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1.  Introduction 


1.1  Background 

The  ability  to  conduct  level  II  interoperability  (receiving  video  feed  from  an  unmanned  aerial 
system  [UAS])  adds  a  new  capability  to  the  AH-64D.  Specifically,  the  UAS  operates  as  a 
remote  sensor  system,  adding  an  additional  sensor  system  to  the  AH-64D  reconnaissance  and 
targeting  systems.  The  Video  from  UAS  for  Interoperability  Teaming  Level  II  (VUIT-2)  system 
was  designed  to  provide  level  II  interoperability  between  the  UAS  and  the  AH-64D.  This  is 
accomplished  by  providing  real-time  streaming  video  from  a  UAS  sensor  to  the  AH-64D  crew 
and  allowing  the  crew  to  re-transmit  that  video  (or  their  aircraft  sensor  video)  to  a  ground  unit 
equipped  with  a  one  system-remote  video  terminal  (OSRVT)  or  to  another  airborne  OSRVT  as 
needed.  The  flow  of  information  between  the  UAS,  AH-64D,  and  OSRVT  is  depicted  in 
figure  1. 

The  U.S.  Army  Research  Laboratory  (ARL)  Human  Research  and  Engineering  Directorate 
(HRED)  conducted  an  AH-64D/UAS  aircrew  workload  assessment  during  February  2008  in 
Huntsville,  AL.  The  purpose  of  the  assessment  was  to  evaluate  AH-64D  aircrew  workload 
during  UAS  level  II  interoperability  (using  VUIT-2)  under  simulated  mission  conditions. 
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1.2  System  Description 

The  major  components  of  the  VUIT-2  system  on  the  AH-64D  are:  a  mast-mounted  C/L-band 
omni-directional  antenna  for  reception  of  UAS  video  and  an  ultra  high  frequency  (UHF)  antenna 
for  reception  of  UAS  data;  an  OSRVT  and  Thermite  computer  to  process  received  video  for 
presentation  on  the  crewmember-selected  multipurpose  display  (MPD);  keyboard  style  control 
that  permits  the  copilot  gunner  (CPG)  to  interface  with  the  VUIT-2  system;  and  a  mini-tactical 
common  data  link  (MTCDL)  system  that  permits  transmission  of  received  UAS  or  aircraft  sensor 
video  to  a  ground  or  airborne  OSRVT.  The  VUIT-2  is  incorporated  as  a  strap-on  system  for 
Block  I  and  II  AH-64Ds  involving  minimal  interface  with  the  current  production  aircraft  bus 
architecture.  The  VUIT-2  software  version  used  during  this  simulation  was  1.7.5. 

The  AH-64D  Apache  is  a  twin-engine,  tandem-seat,  aerial  weapons  platform  built  by  Boeing 
Integrated  Defense  Systems.  Aircraft  armament  includes  a  belly-mounted  slewable  30-mm  chain 
gun,  Hellfire  missiles,  and  2.75-in  aerial  rockets.  The  aircraft  integrated  sensor  suite  includes  a 
mast-mounted  Longbow  fire  control  radar  (FCR)  and  a  nose-mounted  modernized  target 
acquisition  designation  sight/pilot  night  vision  sensor  (MTADS/PNVS).  The  aircraft  displays 
(figure  2)  include  two  MPDs  in  each  cockpit,  the  MTADS  electronic  display  and  control  in  the 
CPG  station,  and  the  integrated  helmet  and  display  sight  system.  The  aircraft  has  a  flight  control 
system  with  a  fully  articulated,  four-bladed  main  rotor  system.  The  flight  control  system  consists 
of  conventional  cockpit  controls:  cyclic,  collective,  and  pedals  connected  mechanically  to 
hydromechanical  actuators  for  the  main  and  tail  rotors;  a  limited  authority  automatic  stabilization 
system;  and  an  electrically  actuated  stabilator. 
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Figure  2.  Depiction  of  VUIT-2  for  the  CPG. 
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1.3  Assessment  Procedure  Overview 

The  workload  simulation  consisted  of  operational  missions  conducted  by  Apache  aircrews  in  the 
AH-64D  Risk  and  Cost  Reduction  Simulator  (RACRS).  The  simulator  was  modified  to 
represent  the  UAS  Level  II  functionality  to  the  maximum  extent  practicable  given  the  maturity  of 
the  Apache  Block  II  design. 

Pilots  received  two  days  of  training  prior  to  the  beginning  of  the  assessment.  The  training 
consisted  of  classroom  instruction  and  hands-on  flight  training  in  the  RACRS.  The  pilots  flew 
the  same  types  of  missions  during  training  that  they  later  flew  during  the  record  trials.  The 
mission  scenario  was  based  on  a  battlefield  environment  simulating  southwest  Asia.  Each 
successive  mission  increased  in  difficulty  in  order  to  impose  progressively  greater  workload  on 
the  pilots.  The  aircrews  performed  specific  Aircrew  Training  Manual  (ATM)  tasks  during  each 
mission.  Each  ATM  task  had  prescribed  conditions  and  standards  to  which  both  crewmembers 
had  to  perform  to  help  ensure  mission  accomplishment.  The  aircrews  also  conducted  missions 
without  VUIT-2  to  establish  a  baseline  for  comparing  workload  during  non-UAS  missions  to 
workload  during  UAS  missions. 

During  the  formal  evaluation,  the  aircrews  performed  Air  Escort  missions.  The  mission 
scenarios  were  developed  by  the  TRADOC  System  Manager,  Reconnaissance  Attack  (TSM  RA) 
office,  Fort  Rucker,  AL.  The  scenarios  were  developed  in  accordance  with  established  aircraft 
tactics,  techniques,  and  procedures  (TTP). 

The  pilots  completed  the  Simulator  Sickness  Questionnaire  (SSQ)  before  and  after  each  flight. 
They  completed  the  Bedford  Workload  Rating  Scale  (BWRS),  Situation  Awareness  Rating 
Technique  (SART),  and  the  UAS-Crewstation  Interface  (UCI)  questionnaire  after  each  mission. 
During  each  mission,  the  CPG  wore  an  eye  tracker.  The  eye  tracker  was  used  to  assess  pilot 
visual  workload.  In  addition  to  the  pilot  data,  subject  matter  experts  (SMEs)  were  used  to 
provide  an  independent  assessment  of  aircrew  workload,  SA,  and  mission  success.  The  SMEs 
completed  an  aircrew  workload,  SA,  and  mission  success  survey  after  each  mission.  After  all 
three  aircrews  completed  the  mission  and  surveys,  they  participated  in  a  mission  debriefing  and 
after-action  review  (AAR). 

During  the  simulation,  aircrew  actions  within  the  cockpit  were  recorded  for  post-test  analysis. 
Video  recordings  of  each  crewstation  and  all  displays  were  kept  as  a  permanent  record.  Log  files 
recorded  all  button  and  switch  activations  and  recorded  which  MPD  display  was  selected  on  each 
of  the  MPDs  during  the  test. 

1.4  Apache  RACRS  Cockpits 

The  RACRS  cockpits  consisted  of  high  fidelity  aircraft  flight  controls  and  displays  (figures  3  and 
4).  The  CPG  utilized  the  TADS  Electronic  Display  and  Control  (TEDAC)  grips  to  control 
Apache  sensors  characteristics  such  as  selecting  sensors,  field  of  view,  azimuth,  elevation,  gain, 
and  level.  These  controls  were  also  selectable  for  adjustment  of  the  UAS  sensor.  The  TEDAC 
and  MPD  displays  were  used  to  monitor  the  sensor  view  from  the  Apache  and/or  the  UAS. 


3 


Figure  3.  Apache  RACRS  simulator  (Camber  Corp). 


Figure  4.  Apache  RACRS  CPG  cockpit  (Camber  Corp). 
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1.5  OneSAF  (One  Semi-Automated  Forces) 

The  OneSAF  simulation  provided  the  ability  to  generate  threats/targets  on  the  battlefield. 
Scenarios  were  developed  that  incorporated  the  detection,  identification,  and  acquisition  of 
simulated  targets.  OneSAF  produced  outputs  on  the  distributed  information  system  (DIS) 
network  enabling  all  attached  simulations  to  receive  and  display  the  entities. 


1.6  UAS  Control  Station 


A  stand-alone  workstation  (figure  5)  was  developed  to  allow  a  UAS  operator  to  independently 
fly  a  UAS  during  the  scenarios.  This  was  a  desktop  computer  with  a  visual  system  representing 
the  same  terrain  location  as  the  Apache  simulation.  A  commercial  joystick  and  keyboard 
provided  user  input  for  UAS  control.  The  operator  was  linked  with  the  Apache  crew  via  audio 
communications  using  the  existing  lab  intercom.  The  UAS  positional  information  was  output 
onto  the  DIS  network,  allowing  the  other  components  to  be  aware  of  its  location. 


Figure  5.  UAS  control  station. 


1.7  AH-64  #2  Control  Station 

A  stand-alone  workstation  (figure  6),  similar  to  the  UAS  control  station,  was  developed  to  allow 
an  operator  to  independently  fly  a  second  AH-64  (wing  man)  during  the  scenarios.  This  was  a 
desktop  computer  with  a  visual  system  representing  the  same  terrain  location  as  the  Apache 
simulation.  A  commercial  joystick  and  keyboard  provided  user  input  for  the  second  AH-64 
control.  The  operator  was  linked  with  the  Apache  crew  via  audio  communications  using  the 
existing  lab  intercom.  The  AH-64  positional  information  was  output  onto  the  DIS  network, 
allowing  the  other  components  to  be  aware  of  its  location. 
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Figure  6.  AH-64  #2  control  station. 


1.8  Terrain  Location 

The  simulator  visual  system  was  configured  to  fly  the  existing  Bagram,  Afghanistan,  visual 
database  (figure  7).  This  is  a  geo-specific  large  gaming  area  built  from  satellite  acquired  high- 
resolution  imagery  and  detailed  terrain  relief.  It  also  contained  ample  terrain  and  cultural 
features  which  supported  the  goals  of  the  assessment. 


Figure  7.  Apache  RACRS  Afghanistan  database  screenshot. 
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2.  Method 


2.1  Data  Collection 

Pilot  workload,  situational  awareness,  crew  coordination,  VUIT-2  crewstation  interface,  switch 
actuations,  simulator  sickness,  visual  gaze  and  dwell  times  (head-eye  tracker),  audio-video,  and 
TTP  data  were  collected  and  analyzed.  These  areas  were  assessed  to  determine  if  (1)  pilot 
workload  was  tolerable  when  interacting  with  an  UAS,  (2)  pilot  workload  in  the  AH-64D  is 
higher,  lower,  or  comparable  to  pilot  workload  when  not  interacting  with  an  UAS,  (3)  pilots  have 
adequate  SA  when  interacting  with  the  UAS,  (4)  the  UAS  control  interface  is  easy  to  understand 
and  navigate,  and  (5)  pilots  experienced  simulator  sickness  symptoms.  The  data  were  being  used 
to  recommend  design  improvements  to  the  UAS  menu  system,  training,  and  associated 
switchology  and  to  refine  TTPs. 

2.2  Pre-  and  Post-Trial  Questionnaires 

ARL  HRED  administered  a  series  of  questionnaires  as  part  of  the  workload  assessment.  The 
questionnaires  were: 

•  Demographics 

•  Simulator  Sickness  Questionnaire  (SSQ) 

•  Workload  (Bedford  Workload  Rating  Scale) 

•  Situational  Awareness  (Situational  Awareness  Rating  Technique) 

•  UAS  Crewstation  Interface  (UCI) 

•  Subject  Matter  Expert  (SME)  Ratings 

2.3  Demographics 

A  demographics  questionnaire  was  used  to  collect  basic  information  on  each  pilot’s  experience 
and  flight  qualifications.  The  demographic  data  documented  the  range  of  pilot  experience  levels 
and  qualifications. 

2.4  Workload 

A  common  definition  of  pilot  workload  is  “the  integrated  mental  and  physical  effort  required  to 
satisfy  the  perceived  demands  of  a  specified  flight  task”  (Roscoe,  1985).  It  is  important  to  assess 
pilot  workload  because  mission  accomplishment  is  related  to  the  mental  and  physical  ability  of 
the  crew  to  effectively  perform  their  flight  and  mission  tasks.  If  pilots  experience  high  workload 
when  they  perform  flight  and  mission  tasks  (e.g.,  control  UASs),  the  tasks  may  be  performed 
ineffectively  or  abandoned. 
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2.4.1  Bedford  Workload  Rating  Scale  (BWRS) 

To  estimate  the  level  of  workload  needed  to  control  the  UAS,  the  pilots  completed  the  BWRS 
immediately  after  each  mission.  The  pilots  rated  the  workload  needed  to  perform  several  UAS, 
flight,  and  mission  tasks.  This  provided  an  overall  assessment  of  the  workload  required  to 
perform  the  missions. 

The  BWRS  has  been  used  extensively  by  the  military,  civil,  and  commercial  aviation 
communities  for  estimating  pilot  workload  (Roscoe  and  Ellis,  1990).  It  requires  pilots  to  rate  the 
level  of  workload  associated  with  a  task  based  on  the  amount  of  spare  capacity  they  feel  they 
have  to  perform  additional  tasks.  Spare  workload  capacity  is  an  important  commodity  for  pilots 
because  they  are  often  required  to  perform  several  tasks  concurrently.  For  example,  pilots  must 
often  perform  flight  tasks  and  navigation  tasks,  and  monitor  radios  within  the  same  time  interval. 
Mission  performance  is  reduced  if  pilots  are  task  saturated  and  have  little  or  no  spare  capacity  to 
perform  other  tasks.  Design  of  the  UCI  should  help  ensure  that  pilots  can  maintain  adequate 
spare  workload  capacity  while  they  control  UASs  and  perform  other  flight  and  mission  tasks. 

2.5  Situation  Awareness  (SA) 

SA  can  be  defined  as  the  pilot’s  mental  model  of  the  current  state  of  the  flight  and  mission 
environment.  A  formal  definition  (Endsley,  1988)  is  “the  perception  of  the  elements  in  the 
environment  within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning,  and  the 
projection  of  their  status  in  the  near  future.”  It  is  important  to  assess  SA  because  it  has  a  direct 
impact  on  pilot  performance.  Good  SA  increases  the  probability  of  good  decisions  and  good 
performance  by  pilots.  The  workload  levels  that  pilots  experience  during  a  mission  affects  their 
SA.  As  pilot  workload  levels  increase,  their  SA  often  decreases.  Design  of  the  crewstation  and 
UAS  UCI  should  ensure  that  pilots  are  able  to  maintain  consistently  high  levels  of  SA  while  they 
control  the  UAS  and  perform  other  flight  and  mission  tasks. 

2.5.1  Situation  Awareness  Rating  Technique  (SART) 

To  estimate  the  level  of  SA  that  they  experienced  during  missions,  the  pilots  completed  the 
SART  scale  after  each  mission.  The  SART  was  developed  as  an  evaluation  tool  for  design  of 
aircrew  systems  (Taylor,  1989)  and  is  composed  of  three  subscales.  The  subscales  are 
Understanding  (U),  Demand  (D),  and  Supply  (S).  Taylor  proposed  that  SA  is  dependent  on  the 
pilot’s  U  (e.g.,  quality  of  information  they  receive),  and  the  difference  between  the  D  (e.g., 
complexity  of  mission)  on  the  pilot’s  resources  and  the  pilot’s  S  (e.g.,  ability  to  concentrate). 
When  D  exceeds  S,  there  is  a  negative  effect  on  U  and  an  overall  reduction  of  SA.  The  formula 
SA  =  U  -  (D  -  S)  is  used  to  derive  the  overall  SART  score.  The  SART  is  one  of  the  most 
thoroughly  tested  rating  scales  for  estimating  SA  (Endsley,  2000). 
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2.6  UAS  Crewstation  Interface  (UCI) 

The  UCI  impacts  crew  workload  and  SA  during  a  mission.  A  UCI  that  is  designed  to  augment 
the  cognitive  and  physical  abilities  of  crews  will  minimize  workload,  enhance  SA,  and  contribute 
to  successful  mission  performance.  To  assess  the  UCI,  the  pilots  reported  any  problems  that 
contributed  to  high  workload  and  low  SA  at  the  end  of  each  mission.  They  also  completed  a 
lengthy  questionnaire  at  the  end  of  their  final  mission.  The  questionnaire  addressed  usability 
characteristics  of  the  UCI. 

2.7  Simulator  Sickness  Questionnaire  (SSQ) 

The  SSQ  was  used  to  assess  any  problems  with  simulator  sickness  in  the  simulator.  Any 
discomfort  felt  by  the  participants  can  adversely  affect  performance  and  how  they  perceive  their 
levels  of  workload  and  SA.  The  SSQ  helped  determine  if  the  visual  effects  of  the  simulator  are  a 
possible  contributor  to  high  workload  and  SA,  rather  than  the  UAS  aircrew  interface  design. 

Simulator  sickness  has  been  defined  as  a  condition  in  which  pilots  suffer  physiological 
discomfort  in  the  simulator,  but  not  while  flying  the  actual  aircraft  (Kennedy  et  al.,  1989).  It  is 
generally  believed  that  simulator  sickness  is  caused  by  a  mismatch  between  the  sensory 
information  (e.g.,  acceleration  cues)  presented  by  the  simulator  and  the  sensory  information 
presented  by  the  primary  aircraft  that  the  pilot  operates.  When  the  sensory  information  presented 
by  the  simulator  does  not  match  the  aircraft,  the  pilot’s  nervous  system  reacts  adversely  to  the 
sensory  mismatch  and  the  pilot  begins  to  experience  discomfort.  Characteristics  of  simulator 
sickness  include  nausea,  dizziness,  drowsiness,  and  several  other  symptoms  (Kennedy  et  ah, 
1989).  It  is  important  to  assess  simulator  sickness  because  the  discomfort  felt  by  pilots  can  be 
distracting.  Pilot  distraction  is  one  of  the  operational  consequences  of  simulator  sickness  listed 
by  Crowley  (1987).  If  pilots  are  distracted  by  the  discomfort  they  feel  during  missions,  their 
performance  is  likely  to  suffer.  Additionally,  the  discomfort  could  influence  the  perceived  levels 
of  workload  and  SA  that  the  pilots  experience  during  a  mission. 

The  SSQ  (Kennedy  et  ah,  1993)  is  a  checklist  of  16  symptoms.  The  16  symptoms  are 
categorized  into  three  subscales.  The  subscales  are  Oculomotor  (e.g.,  eyestrain,  difficulty 
focusing,  blurred  vision),  Disorientation  (e.g.,  dizziness,  vertigo),  and  Nausea  (e.g.,  nausea, 
increased  salivation,  burping).  The  three  subscales  are  combined  to  produce  a  Total  Severity 
score.  The  Total  Severity  score  is  an  indicator  of  the  overall  discomfort  that  the  pilots 
experienced  during  the  mission. 

2.8  Switch  Actuations 

The  RACRS  was  instrumented  to  log  switch  actuations  to  help  determine  whether  there  were 
tasks  (e.g.,  UAS  menu  navigation)  that  need  to  be  streamlined  because  of  excessive  switch 
manipulation. 
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2.9  Audio-Video  Collection 

Audio,  video,  and  digital  photography  data  collection  was  recorded  during  the  simulation.  Video 
cameras  were  used  in  the  front  seat  and  back  seat  of  the  RACRS  during  the  trials  to  record  pilot 
actions.  All  voice  communications  that  occurred  over  the  interphone  communications  system 
(ICS)  were  recorded  and  dubbed  into  the  video  recordings. 

2.10  After- Action  Reviews  (AARs) 

Daily  AARs  were  conducted  by  the  AH-64D  Product  Manager’s  Office  (PMO)  to  help  analyze 
workload,  SA,  and  document  TTP  insights  about  the  optimal  employment  of  UASs. 

2.11  Subject  Matter  Experts  (SMEs) 

SMEs  observed  each  mission  and  rated  crew  workload,  crew  SA,  crew  coordination,  and  mission 
success.  The  two  SMEs  were  both  CW4  Apache  pilots,  and  each  had  over  3000  hours  of  flight 
experience.  The  SMEs  were  familiar  with  UAS  Concept  of  Operations  (CONOPS).  The  SMEs 
provided  an  independent  assessment  of  the  workload  and  SA  levels  experienced  by  the  crews. 
They  helped  identify  whether  problems  with  crew  workload  or  crew  SA  contributed  to  lack  of 
mission  success.  SME  personnel  observed  the  missions  using  a  suite  of  monitors  that  showed  all 
crewstation  displays  in  the  RACRS.  They  also  listened  to  all  audio  communications  between 
crewmembers,  aircraft,  ground  control  station,  ground  commander,  and  the  tactical  operations 
center  during  the  missions.  Several  large  displays  provided  real-time  status  of  the  location  of  the 
aircraft,  UAS,  friendly  forces,  and  enemy  forces  to  the  SMEs. 

2.12  Evaluation  Design 

While  the  evaluation  was  operational  in  nature  rather  than  experimental,  multiple  variables  were 
controlled  in  order  to  maximize  the  validity  of  the  conclusions  regarding  the  areas  of  evaluation. 
Table  1  summarizes  the  variables  that  were  controlled  during  the  simulation. 


Table  1.  Variables  that  were  controlled  during  the  simulation. 


Factor 

Control 

Conditions 

Mission 

Constant 

Air  Assault 

Flight  profile 

Tactically  varied 

Nap  of  the  Earth  (NOE),  contour 

Light  conditions 

Constant 

Day 

Scenario 

Constant 

Southwest  Asia 

Crew 

Constant 

Maximize  crew  familiarity 

Seat  position 

Varied 

Front,  back 

Flight  uniform 

Constant 

Air  Warrior  Gen  3  combat-basic 
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3.  Instrumentation 


3.1  Switch  Actuations 

When  initiated,  the  data  collection  software  collected  switch  actuations  at  a  10  Hz  rate.  The 
collected  data  were  time-stamped  and  written  to  a  comma-delimited  log  file.  The  log  file  was 
then  post-processed  to  an  Excel  spreadsheet  format.  The  data  collection  software  provided  an 
interface  to  allow  the  operator  to  initiate  and  cease  collection  during  an  executed  run. 

3.2  Overhead  Cockpit  Cameras 

An  overhead  camera  was  mounted  in  the  front  seat  and  a  small  camera  was  mounted  on  the 
glareshield  in  the  back  seat  to  record  pilot  actions.  This  aided  in  determining  what  the  pilots 
were  doing  during  different  phases  of  the  mission.  The  cameras  had  a  time  stamp  added  so  it 
could  be  compared  with  other  data  collection  material. 

3.3  Head  and  Eye  Tracker  System 

A  head-eye  tracking  system  from  Applied  Science  Laboratories  (ASL)  was  used  to  measure 
visual  gaze  and  dwell  times,  and  to  help  assess  visual  workload  of  the  pilot  (CPG)  who 
controlled  the  UAS.  Data  obtained  via  the  eye  tracker  helped  determine  if  pilots  were  required 
to  spend  too  much  time  looking  at  the  crewstation  displays  to  perform  tasks.  Data  provided  by 
the  eye  tracker  was  summarized  in  a  graphical  format.  The  format  depicts  the  amount  of  time 
that  each  pilot  spent  looking  at  the  crewstation  displays  and  controls  vs.  the  out-the-window 
visual  scene.  The  ASL  501  head-eye  tracker  (figure  8)  has  been  used  extensively  by  the  military 
and  industry  and  presents  no  known  health  hazards  to  the  wearer.  The  system  was  mounted  onto 
the  pilot’s  flight  helmet  (front  seat  only)  via  the  Night  Vision  Goggle  (NVG)  mount.  Head-eye 
tracker  results  complemented  the  questionnaire  results  and  provide  an  increased  understanding  of 
crewstation  workload  and  UCI. 

3.4  Data  Analysis,  Limitations,  and  Pilot  Demographics 
3.4.1  Data  Analysis 

Pilot  responses  to  the  BWRS,  SART,  SSQ,  and  UCI  questionnaires  were  analyzed  with  means 
and  percentages.  Their  responses  to  the  BWRS,  SART,  and  SSQ  were  further  analyzed  with  the 
Wilcoxon  Signed  Rank  Test  (WSRT)  to  compare  the  ratings  between  the  CPG  and  pilot  and 
between  ratings  for  the  VUIT-2  missions  vs.  non-VUIT-2  missions  (MTADS  only)  to  determine 
if  the  differences  were  statistically  significant  at  p  =  0.05. 
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Figure  8.  Eye  tracker,  pupil/camera  monitors,  and  control  panel  interface. 


The  eye  tracker  data  were  summarized  by  calculating  the  total  percentage  of  fixations  that 
occurred  for  the  different  areas  of  interest  (AOI).  Six  AOIs  were  created  for  the  CPG:  right 
MPD,  left  MPD,  TEDAC,  keyboard,  and  kneeboard.  Visual  gaze  and  dwell  times  were  also 
recorded  for  out-the-window.  A  final  category,  called  “Other’,  captured  eye  fixations  not 
focused  on  a  specific  AOI. 

3.4.2  Evaluation  Limitations 

The  primary  limitations  included  the  small  sample  size  of  pilots  (N=8)  who  participated  in  the 
crewstation  simulation  assessment,  the  limited  training  they  received  (two  days),  and  mix  of  AH- 
64D  software  versions  in  the  RACRS. 

These  limitations  are  typical  when  constrained  by  time  and  funding,  and  when  replicating  a 
complex  aviation  system  in  a  simulator.  However,  the  information  and  data  listed  in  the  Results 
and  Conclustion  sections  of  this  report  should  be  interpreted  based  on  these  limitations. 
Additional  data  should  be  collected  during  future  simulations  and  tests  to  augment  and  expand 
the  findings  contained  in  this  report. 

3.4.3  Participants 

Eight  AH-64D  pilots  participated  in  the  assessment.  Six  pilots  were  assigned  to  the  3-3  Aviation 
Regiment,  Fort  Stewart,  GA;  one  pilot  was  assigned  to  the  Directorate  of  Evaluation  and 
Standardization,  Fort  Rucker,  AL;  and  one  pilot  was  assigned  to  the  AH-64D  Apache  Longbow 
Program  Management  Office,  Redstone  Arsenal,  AL.  All  pilots  held  the  rank  of  warrant  officer 
(CW2  =  2  pilots,  CW3  =  2  pilots,  CW4  =  4  pilots).  Five  pilots  were  Flight  Activity  Category 
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(FAC)  1,  two  pilots  were  FAC  2,  and  one  pilot  was  FAC  3.  Seven  pilots  were  Readiness  Level 
(RL)  1  and  one  pilot  was  RL  3.  They  represented  a  broad  range  of  experience  with  total  flight 
hours  from  500  to  3800  hours.  The  pilot  demographics  are  listed  in  table  2. 

Table  2.  Pilot  demographics  (N  =  8). 


Summary  of 
Demographic 
Characteristics 

Age 

(yrs) 

Flight  Hours 
in  AH-64D 

Total  Flight  Hours 
in  Army  Aircraft 

Mean 

37 

1050 

2428 

Median 

37 

1300 

2525 

Range 

29-43 

150-1700 

500-3800 

4.  Results 


4.1  Crew  Workload 

4.1.1  Average  Bedford  Workload  Ratings  for  Flight  and  Mission  Tasks 

The  average  overall  Bedford  workload  rating  for  VUIT-2  missions  was  3.3  for  the  CPG  and  2.6 
for  the  pilot  (PI)  (figure  9).  These  ratings  indicate  that  the  CPGs  and  Pis  typically  felt  that 
workload  was  tolerable  for  the  tasks,  and  they  had  enough  spare  mental  capacity  for  all  desirable 
additional  tasks.  The  difference  in  workload  ratings  between  the  CPG  and  PI  was  statistically 
significant  (WSRT,  Z  =  -4.109,  p  <  0.01).  This  suggests  that  the  pilots  firmly  believed  that 
workload  levels  were  somewhat  higher  for  the  CPG  than  the  PI  during  most  missions. 


Pilot  and  CPG  Bedford  Workload  Ratings 


VUIT-2  Missions  Non-VUIT-2  Missions 


Figure  9.  PI  and  CPG  workload  ratings. 
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The  overall  Bedford  workload  rating  for  the  non-VUIT-2  missions  was  3.0  for  the  CPG.  This 
rating  indicates  that  the  CPGs  typically  believed  that  workload  was  tolerable,  and  they  had 
enough  spare  mental  capacity  for  all  desirable  additional  tasks.  The  overall  Bedford  workload 
rating  for  the  non-VUIT-2  missions  was  3.7  for  the  PI.  This  rating  indicates  that  the  Pis 
typically  believed  that  workload  was  tolerable,  but  they  had  insufficient  spare  capacity  for  easy 
attention  to  additional  tasks.  The  difference  in  workload  ratings  between  the  CPG  and  PI  was 
not  statistically  significant  (WSRT,  Z  =  -1.684,  p  =  0.09).  This  suggests  that  pilots  perceived 
that  workload  levels  were  comparable  for  the  CPG  and  the  PI  during  most  missions.  The 
differences  in  workload  ratings  between  VUIT-2  missions  and  non-VUIT-2  missions  was  not 
statistically  significant  for  the  CPG  (WSRT,  Z  =  -0.674,  p  =  0.500),  nor  statistically  significant 
for  the  PI  (WSRT,  Z  =  -0.502,  p  =  0.616).  The  majority  of  pilots  commented  that  the  workload 
levels  they  experienced  during  VUIT-2  missions  were  comparable  to  workload  levels  they 
experienced  during  missions  using  only  the  MTADS.  They  reported  that  having  to  manage  an 
additional  sensor  (UAS  sensor)  increased  their  overall  task  workload,  but  the  SA  provided  by  the 
UAS  sensor  typically  decreased  the  workload  required  to  detect  and  engage  targets.  They  stated 
that  (1)  it  was  easier  for  them  to  detect  and  engage  targets  using  the  UAS  sensor  because  of  the 
steep  visual  aspect  angle  (‘God’s  Eye’  view)  that  the  UAS  sensor  provided,  compared  to  the 
shallower  visual  aspect  angle  that  the  MTADS  typically  provides  during  missions,  (2)  the  SA 
provided  by  the  UAS  sensor  reduced  the  time  required  to  detect  and  engage  targets,  and  (3) 
while  they  had  more  tasks  to  complete  because  of  the  additional  sensor,  they  had  more  time  to 
complete  target  detection  and  engagement  tasks  because  of  the  greater  stand-off  range  that  the 
UAS  sensor  provided  vs.  the  stand-off  range  that  the  MTADS  provides  during  missions. 

4.1.2  CPG  Task  Shedding 

The  CPGs  reported  that  there  were  very  few  tasks  that,  due  to  high  mission  workload,  they  had 
to  ask  the  Pis  to  perform.  ARL  personnel  observed  only  a  few  instances  when  the  CPG  asked 
the  PI  to  perform  a  task  because  he  was  experiencing  high  workload.  However,  the  CPGs 
occasionally  shed  tasks  such  as  responding  to  radio  calls.  There  was  one  instance,  due  to  high 
mission  workload,  when  the  CPG  was  unable  to  acquire  a  target  and  perform  normal  engagement 
tasks.  In  this  instance,  the  PI  utilized  the  30-mm  gun  to  engage  a  target  because  the  aircraft  was 
in  close  range  to  the  threat. 

4.1.3  SME  Ratings  of  Aircrew  Coordination 

The  SMEs  provided  ratings  (figure  10)  of  how  well  the  crews  performed  aircrew  coordination 
tasks  (e.g.,  positive  communication)  per  Training  Circular  1-210.  The  SMEs  rated  aircrew 
coordination  during  VUIT-2  missions  as  ‘Excellent’  (15%),  ‘Good’  (31%),  or  ‘Average’  (54%). 
The  SMEs  rated  aircrew  coordination  during  non-VUIT-2  missions  as  ‘Excellent’  (17%),  ‘Good’ 
(33%),  or  ‘Average’  (50%). 
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Figure  10.  SME  ratings  for  PI  and  CPG  aircrew  coordination. 


4.1.4  SME  Workload  Ratings 

SMEs  provided  an  overall  Bedford  workload  rating  for  each  mission  that  they  observed.  The 
average  SME  Bedford  workload  rating  (figure  1 1)  was  3.9  for  the  CPG  and  2.5  for  the  PI  during 
the  VUIT-2  missions.  The  average  SME  Bedford  workload  rating  was  3.3  for  the  CPG  and  4.2 
for  the  PI  during  the  non- VUIT-2  missions.  These  ratings  indicate  that  the  SMEs  believed  that 
(1)  workload  was  tolerable  for  the  CPG  and  PI  during  VUIT-2  missions,  but  the  CPG  had 
insufficient  spare  capacity  for  easy  attention  to  additional  tasks;  and  (2)  workload  was  tolerable 
for  the  CPG  and  PI  during  non- VUIT-2  missions,  but  the  PI  had  insufficient  spare  capacity  for 
easy  attention  to  additional  tasks.  The  difference  in  SME  workload  ratings  between  the  CPG  and 
PI  was  statistically  significant  for  the  VUIT-2  missions  (WSRT,  Z  =  -2.555,  p  =  0.01 1),  but  not 
statistically  significant  for  the  non- VUIT-2  missions  (WSRT,  Z  =  -0.447,  p  =  0.655). 


SME  Bedford  Workload  Ratings 

10  V 


VUIT-2  Missions  Non-VUIT-2  Missions 


Figure  1 1.  SME  Bedford  workload  ratings. 
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In  previous  simulations  and  operational  tests  that  ARL  has  helped  conduct,  SMEs  have  typically 
rated  pilot  workload  higher  than  the  ratings  provided  by  the  pilots.  SMEs  have  more  information 
(e.g.,  location  of  all  threat  and  friendly  vehicles)  available  to  them  to  assess  pilot  performance 
and  workload  than  the  information  that  the  pilots  have  available  to  them.  SMEs  are  often  more 
aware  of  pilot  mistakes  (that  may  be  attributable  to  workload)  than  the  pilots.  The  additional 
information  that  the  SMEs  have  likely  results  in  a  more  critical  assessment  of  pilot  performance 
and  workload. 

4.1.5  Visual  Workload 

Figure  12  shows  the  average  percentage  of  time  that  the  CPGs  were  visually  focused  on  each 
AOI  during  the  VUIT-2  missions.  The  CPGs  were  visually  focused  on  the  right  MPD  for  13% 
of  the  time  during  missions.  They  were  visually  focused  on  the  left  MPD  for  17%  of  the  time 
and  the  TED  AC  (MTADS  sensor)  for  48%.  It  is  interesting  to  note  that  the  CPGs  typically  spent 
only  3%  of  the  time  visually  focused  out-the-window  during  missions.  Figure  13  shows  the 
average  percentage  of  time  that  the  CPGs  were  visually  focused  on  each  AOI  during  the  non- 
VUIT-2  missions.  The  CPGs  were  visually  focused  on  the  right  MPD  for  10%  of  the  time 
during  missions.  They  were  visually  focused  on  the  left  MPD  for  4%  of  the  time  and  the 
TEDAC  for  73%. 
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Figure  12.  CPG  visual  gaze  and  dwell  times  during  VUIT-2  missions. 
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Figure  13.  CPG  visual  gaze  and  dwell  times  during  non-VUIT-2  missions. 

A  small  camera  was  mounted  on  the  glareshield  in  the  rear  cockpit  so  that  ARL  personnel  could 
observe  how  much  time  the  Pis  spent  visually  focused  inside  versus  outside  the  aircraft.  ARL 
personnel  observed  that  the  Pis  were  visually  focused  outside  the  aircraft  approximately  75%  of 
the  time  and  inside  25%  of  the  time  during  the  VUIT-2  missions  and  non-VUIT-2  missions 
(figure  14).  The  Pis  confirmed  the  observations  made  by  ARL  personnel  during  post-mission 
discussions. 

During  the  VUIT-2  missions  and  non-VUIT-2  missions,  the  CPGs  were  not  able  to  maintain 
visual  focus  outside  the  aircraft  to  assist  with  navigation  (e.g.,  identification  of  terrain  features), 
local  security,  terrain  flight,  etc.  For  96+%  of  the  time,  the  CPGs  were  visually  focused  inside 
the  aircraft  performing  (mostly)  target  detection  and  engagement  tasks.  It  should  be  cautioned 
that  the  simulation  was  conducted  with  a  small  sample  size  of  pilots,  the  pilots  did  not  get  all  of 
the  peripheral  visual  cues  that  they  would  in  an  aircraft,  and  pilots  knew  that  they  would  not 
suffer  harm  if  they  crashed  the  simulator. 
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In  The  Cockpit 


Out-The-Window 
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Observing  Pilot  Visual 
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25% 


Figure  14.  PI  visual  dwell  times  estimates  for  all  missions. 

4.1.6  Comparison  of  Eye  Tracker  Data 

Table  3  shows  a  comparison  of  AH-64D,  Armed  Recon  Helicopter  (ARH)  and  UH-60M  eye 
tracker  data  for  Visual  Flight  Rules  (VFR)  flight  during  various  aircraft  simulations.  These 
simulations  were  observed  by  ARL-HRED  personnel  and  include  workload  assessments,  limited 
user  tests,  and  crewstation  evaluations.  While  the  aircraft,  missions,  training  and  personnel 
experience  levels  were  different  for  each  simulation  evaluation,  it  is  interesting  to  note  the 
differences  in  visual  gaze  and  dwell  times  for  each  evaluation. 

Table  3.  Comparison  of  eye  tracker  results  for  AH-64D,  UH-60M,  and  ARH-70  simulations. 


AH-64DWUIT- 
2  Workload 
Assessment 

AH-64D/UAS 

Workload 

Assessment 

(non-VUIT-2) 

AH-64D 
Workload 
Assessment 
(Block  III) 

UH-60M  Limited 
User  Test 

ARH-70  HFE- 
CAAS  Evaluation 

Flying 

Pilot3 

(%) 

CPG 

(%) 

Flying 

Pilot3 

(%) 

Non 

Flying 

Pilot 

(%) 

Flying 

Pilot3 

(%) 

Non 

Flying 

Pilot 

(%) 

Flying 

Pilot 

(%) 

Non 

Flying 

Pilot 

(%) 

Flying 

Pilot 

(%) 

Non 

Flying 

Pilot 

(%) 

Outside 

75 

3 

75 

4 

75 

6 

86 

28 

61 

7 

Inside 

25 

97 

25 

96 

25 

94 

14 

72 

39 

93 

aEstimate  from  watching  PI  gaze  and  dwell  times  with  video  camera  during  missions  (non-eye  tracker). 
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4.2  Crew  Situation  Awareness 


4.2.1  Situation  Awareness  Ratings 

The  overall  SART  scores  provided  by  the  pilots  were  19.0  for  the  CPG  and  2 1 .3  for  the  PI 
(figure  15)  during  the  VUIT-2  missions.  These  scores  indicate  that  the  CPG  and  PI  felt  they  had 
moderate  levels  of  overall  SA  during  the  missions.  The  difference  between  SART  scores  for  the 
CPG  and  PI  was  not  statistically  significant  (WSRT,  Z  =  -0.491,  p  =  0.624).  The  SART  scores 
for  non- VUIT-2  missions  were  19.6  for  the  CPG  and  18.0  for  the  PI  (figure  15).  The  difference 
between  SART  scores  for  the  CPG  and  PI  was  not  statistically  significant  for  the  non-VUIT-2 
missions  (WSRT,  Z  =  -1.633,  p  =  0.102).  The  difference  between  SART  scores  for  the  VUIT-2 
missions  and  non-VUIT-2  missions  was  not  statistically  significant  for  the  CPG  (WSRT,  Z  = 
-0.535,  p  =  0.593)  or  for  the  PI  (WSRT,  Z  =  -1.342,  p  =  0.180).  The  pilots  stated  that  they  had 
higher  SA  during  VUIT-2  missions  (vs.  non-VUIT-2)  mostly  because  of  the  “God’s  Eye”  view 
that  the  UAS  sensor  video  provided  during  missions.  The  UAS  sensor  video  also  gave  the  pilots 
good  SA  earlier  in  the  mission  (vs.  non-VUIT-2  missions)  because  they  often  received  the  video 
prior  to  (or  just  after)  take-off  of  their  aircraft. 


SART  Scores  For  VUIT-2  Missions  vs. 


Non-VUIT-2  Missions 


VUIT-2  Missions  Non-VUIT-2  Missions 


Figure  15.  Overall  SART  scores  for  PI  and  CPG. 

The  pilots  reported  that  they  had  moderate  levels  of  SA  of  most  of  the  battlefield  elements 
(appendix  C)  during  both  VUIT-2  and  non-VUIT-2  missions.  The  battlefield  elements  included 
location  of  enemy  and  friendly  units,  location  of  own  ship,  location  of  cultural  features  (e.g., 
bridges),  and  route  information  (e.g.,  waypoints).  However,  there  were  instances  during  VUIT-2 
missions  and  non-VUIT-2  missions  when  the  aircraft  flew  near  the  target(s)  and  missiles  were 
fired  outside  of  the  aircraft/UAS  constraints.  This  was  likely  caused  by  the  lack  of  extensive 
training  and  experience  with  the  UAS  and  the  need  for  improved  cueing  symbology  to  help 
pilots  understand  where  both  their  aircraft  and  the  UAS  are  located  in  reference  to  the  targets. 
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4.2.2  SME  Situation  Awareness  Ratings 

The  SMEs  provided  an  independent  assessment  of  aircrew  SA  based  on  the  scale  shown  in 
table  4.  The  SMEs  mean  SA  rating  for  aircrews  during  VUIT-2  missions  was  2.08.  The  SMEs 
mean  SA  rating  for  non- VUIT-2  missions  was  3.0.  This  indicates  that  the  SMEs  perceived  that 
the  aircrews  typically  had  adequate  levels  of  SA  during  the  VUIT-2  and  non- VUIT-2  missions. 


Table  4.  SME  situation  awareness  rating. 


SME  Situation  Awareness  Ratings 

1 

Crew  was  consistently  aware  of  all  entities  on  the  battlefield  as  well  as  the  status 
of  their  aircraft. 

2 

Crew  was  aware  of  the  battlefield  and  their  own  ship  with  minor  or  insignificant  * 
variation  between  perception  and  reality. 

3 

Crew  was  aware  of  the  battlefield  and  their  own  ship.  Variation  between  reality  * 
and  perception  did  not  significantly  impact  mission  success. 

4 

SA  needs  improvement.  Lack  of  SA  had  some  negative  effect  on  the  success  of 
the  mission. 

5 

Lack  of  SA  caused  mission  failure. 

Mean  Rating 
(VUIT-2) 
2.08 

(SD  =  0.95) 


Mean  Rating 
(Non-VUIT-2) 
3.00 

(SD=  1.55) 


The  SMEs  noted  that  there  were  instances  when  the  aircraft  flew  near  the  target(s)  and  missiles 
were  fired  outside  of  the  aircraft/UAS  constraints.  During  the  non- VUIT-2  missions,  crew 
members  occasionally  had  difficulty  identifying  friendly  targets  using  the  MTADS.  The 
misidentification  of  targets  resulted  in  fratricide  and  increased  workload  while  both  pilots  tried  to 
determine  whether  the  target  was  friendly. 

4.2.3  SME  Ratings  of  Mission  Success  and  Mission  Objectives 

At  the  end  of  each  mission,  SMEs  rated  whether  the  mission  was  a  success  or  failure.  The 
criteria  that  the  SMEs  used  to  rate  mission  success  or  failure  was  whether  the  aircrew  completed 
most  or  all  of  their  mission  objectives  and  did  not  get  shot  down  or  crash.  The  SMEs  rated  all  of 
the  VUIT-2  missions  as  “successful”  (figure  16).  The  SMEs  rated  17%  of  the  non- VUIT-2 
missions  as  failures  or  incomplete.  This  was  due  to  a  fratricide  resulting  from  a  misidentification 
of  a  friendly  entity.  They  also  rated  whether  the  aircrew  completed  their  mission  objectives. 

The  mission  objectives  were  given  to  the  pilots  during  the  pre-mission  brief  and  are  listed  in 
appendix  G.  The  SMEs  believed  that  the  aircrews  completed  their  objectives  for  100%  of  the 
VUIT-2  missions  and  83%  of  the  non- VUIT-2  missions. 
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4.3  UAS  Crewstation  Interface  (UCI) 

Most  pilots  reported  that  the  UCI  was  usable,  but  needed  improvements  to  enhance  overall 
effectiveness.  The  pilots  reported  some  difficulty  operating  the  buttons  on  the  VUIT-2  keypad, 
and  that  a  backspace  or  clear  button  was  needed  to  reduce  workload  and  the  amount  of  time 
needed  during  data  entry  procedures  to  correct  any  errors.  Most  CPGs  reported  that  the  UCI 
contributed  to  higher  workload  during  the  missions  due  to  difficult  data  entry  procedures  and 
map  symbology.  The  CPGs  also  occasionally  forgot  the  steps  required  for  operating  the  UAS 
interface  during  set  up  and  target  tracking.  All  CPGs  reported  that  with  more  experience  and 
training,  they  would  be  able  to  remember  the  steps  required  for  operating  the  UAS  interface. 
CPGs  reported  that  the  map  symbology  for  the  UAS  was  difficult  to  read  and  interpret. 
Suggestions  were  made  to  change  the  color  and  scale  of  the  icons,  along  with  modifying  the 
manner  in  which  the  direction  of  the  UAS  sensor  is  displayed  on  the  map.  Several 
improvements  that  the  pilots  recommended  be  made  to  the  UCI  to  increase  usability  and 
decrease  workload,  and  the  time  required  to  complete  tasks,  are  listed  in  appendix  E. 

4.4  Switch  Actuations 

Switch  actuations  were  recorded  to  help  determine  whether  there  were  CPG  tasks  (e.g.,  UAS 
menu  navigation)  that  need  to  be  streamlined  because  of  excessive  switch  actuations.  The  switch 
actuations  are  summarized  in  appendix  F.  The  CPGs  made  an  average  of  316  switch  actuations 
per  UAS  mission.  The  CPGs  made  an  average  of  53  VUIT-2  specific  button  presses  per  mission. 
The  total  average  combined  number  of  button  presses  for  VUIT-2  missions  was  369.  This 
equates  to  approximately  7-8  switch  actuations  per  minute,  or  one  switch  actuation  per  8  s  for 
each  mission.  The  switch  actuations  were  often  clumped  together  within  specific  time  intervals. 
Non-VUIT-2  missions  averaged  525  button  presses  per  mission.  This  equates  to  approximately 
1 1  button  presses  per  minute,  or  one  switch  actuation  every  5  s  interval  for  each  mission.  The 
rise  in  the  number  of  switch  actuations  from  VUIT-2  missions  to  non- VUIT-2  missions  was 
mostly  due  to  the  increased  use  of  the  MTADS  FOV  switch.  The  pilots  were  forced  to  use  the 
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MTADS  to  search  for  targets  rather  than  the  UAS  video  feed,  causing  an  increase  in  the  amount 
of  button  presses  required  to  search  and  locate  targets.  The  switches  that  were  actuated  most 
often  by  the  CPG  during  both  VUIT-2  and  non-VUIT-2  missions  were  the  MTADS  FOV  select 
switch,  right-hand  grip  slave  select  switch,  left-hand  grip  tracker  switch,  right-hand  grip  laser 
trigger,  right  MPD  bezel  buttons,  and  the  weapon  trigger.  The  pilots  reported  that  they  did  not 
think  that  the  number  of  switch  actuations  per  mission  was  excessive.  Many  of  the  switch 
actuations  were  momentary  actuations  of  the  MTADS  FOV  select  switch  and  right-hand  grip 
slave  select  switch. 

Table  5  summarizes  the  number  of  MPD  page  changes  per  mission.  The  average  number  of 
right  and  left  MPD  page  changes  was  50  for  the  CPG  and  48  for  the  PI  per  VUIT-2  mission.  The 
average  number  of  right  and  left  MPD  page  changes  during  non-VUIT-2  missions  was  39  for  the 
CPG  and  24  for  the  PI  (table  6).  The  Tactical  Situation  Display  (TSD)  and  Tactical  Common 
Data  Link  (TCDL)  pages  were  the  pages  most  often  displayed  by  the  CPG  and  PI  during 
missions. 


Table  5.  MPD  page  changes  during  the  VUIT-2  missions. 


No.  of  Page  Changes  per  Mission  (VUIT-2  Missions) 

Mission 

CPG 

(Right  MPD) 

CPG 

(Left  MPD) 

Pilot 

(Right  MPD) 

Pilot 

(Left  MPD) 

1 

59 

47 

40 

42 

2 

34 

20 

9 

76 

3 

2 

35 

27 

8 

4 

40 

2 

14 

30 

5 

29 

26 

11 

20 

6 

28 

0 

0 

6 

7 

24 

15 

28 

44 

8 

31 

1 

13 

14 

Average 

30.875 

18.25 

17.75 

30 

Std.  dev. 

15.93 

17.22 

12.87 

23.52 

Min 

2 

0 

0 

6 

Max 

59 

47 

40 

76 

Table  6.  MPD  page  changes  during  the  non-VUIT-2  missions. 


No.  of  Page  Changes 

per  Mission(Non-VUIT-2  Missions) 

Mission 

CPG 

(Right  MPD) 

CPG 

(Left  MPD) 

Pilot 

(Right  MPD) 

Pilot 

(Left  MPD) 

9 

22 

18 

5 

21 

10 

21 

2 

8 

42 

1 1 

14 

37 

19 

13 

12 

24 

16 

1 1 

0 

Average 

20.25 

18.25 

10.75 

19 

Std.  dev. 

4.35 

14.38 

6.02 

17.61 

Min 

14 

2 

5 

0 

Max 

24 

37 

19 

42 
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4.5  Top  Improvements  Recommended  by  Pilots 

The  pilots  recommended  that  several  improvements  be  made  to  the  UCI  and  UAS  to  increase 
usability  and  decrease  workload  and  the  time  required  to  complete  tasks.  Following  are  the  most 
significant  improvements  that  the  pilots  recommended  be  made  to  improve  UAS  employment. 
Examples  for  each  improvement  are  listed  in  appendix  E. 

•  Need  the  ability  to  record  MTADS  and  VUIT-2  video  and  audio  communications  without 
switching  one  recorder  off. 

•  UAV  icon  colors  should  be  changed  to  make  them  more  easily  recognizable  while 
scanning. 

•  Map  needs  increased  fidelity  and  proper  scaling  of  icons  and  symbology. 

•  UAV  icons  should  be  integrated  into  the  current  TSD  page. 

•  Improve  functionality  of  the  VUIT-2  keypad,  to  include  a  backspace  function  and  enter 
key. 

•  Need  battlefield  graphics  to  aid  with  remote  laser  designation  during  target  engagements. 

4.6  Simulator  Sickness 

Pilots  reported  that  they  typically  experienced  very  mild  to  mild  simulator  sickness  symptoms 
during  the  evaluation.  The  overall  mean  Total  Severity  (TS)  score  (post  mission)  for  the  pilots 
was  4.98  (table  7).  The  mean  TS  score  for  the  CPGs  was  7.79  and  the  mean  TS  score  for  the  Pis 
was  2. 18.  The  difference  between  the  pre-flight  SSQ  and  the  post-flight  SSQ  scores  for  the  CPG 
was  not  statistically  significant  (WSRT,  Z  =  -0.768,  p  =  0.461).  The  difference  between  the  pre¬ 
flight  SSQ  and  the  post-flight  SSQ  scores  for  the  PI  was  also  not  statistically  significant  (WSRT, 
Z  =  0.000,  p  =  1.000).  Based  on  the  categorization  of  simulator  sickness  symptoms  proposed  by 
Kennedy  et  al.  (2002)  (table  8),  the  pilots  experienced  “minimal”  simulator  sickness  symptoms 
during  the  missions. 

Table  7.  Simulator  sickness  questionnaire  (SSQ)  ratings. 


Condition 

Nausea 

Subscale 

Oculomotor 

Subscale 

Disorientation 

Subscale 

Total  Severity 
Score  (Mean) 

Pre-mission  (CPG) 

3.97 

6.31 

4.64 

5.92 

Post-mission  (CPG) 

3.97 

7.58 

9.28 

7.79 

Pre-mission  (PI) 

1.59 

3.15 

0 

2.18 

Post-mission  (PI) 

2.38 

2.52 

0 

2.18 

Pre-mission  CPG  and  PI 

2.78 

4.73 

2.32 

4.05 

Post-mission  CPG  and  PI 

3.18 

5.05 

4.64 

4.98 
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Table  8.  Categorization  of  simulator  sickness  symptoms. 


SSQ  Total  Score 

Categorization 

0 

No  symptoms 

<5 

Negligible  symptoms  (PI) 

5-10 

Minimal  symptoms  (CPG) 

10-15 

Significant  symptoms 

15-20 

Symptoms  are  a  concern 

>20 

A  problem  simulator 

Note:  Categorization  of  symptoms  based  on  central  tendency  (mean  or  median) 
using  military  aviation  personnel  in  each  simulator  (Kennedy,  2002). 


4.7  Comparison  of  RACRS  Simulator  SSQ  Scores  to  Other  Helicopter  Simulators 

To  assess  whether  the  SSQ  ratings  provided  by  the  pilots  during  the  assessment  were  similar  or 
different  to  ratings  obtained  in  other  helicopter  simulators,  the  mean  SSQ  scores  for  the  RACRS 
simulator  were  compared  to  the  mean  SSQ  scores  for  several  other  helicopter  simulators  (table  9). 
The  other  helicopter  simulators  were  the  AH-64A  (Army  Research  Institute,  non-motion 
simulator),  Armed  Reconnaissance  Helicopter  (ARH-70),  S-3H,  CH-46E,  CH-56D,  CH-56F, 
Sikorsky  RAH-66  Engineering  Development  Simulator  (EDS),  RAH-66  Comanche  Portable 
Cockpit  (CPC),  and  the  simulator  used  during  the  UH-60M  for  the  Early  User  Demo  (EUD), 
RACRS  (UAS)  and  Limited  Early  User  Evaluation  (LEUE).  In  comparison,  the  RACRS 
induced  fewer  simulator  sickness  symptoms  than  the  other  helicopter  simulators  listed  in  table  9. 

Based  on  pilot  SSQ  ratings,  observation  by  ARL  HRED  personnel  during  missions,  feedback 
during  post  mission  interviews,  and  comparison  of  SSQ  ratings  with  ratings  from  other 
helicopter  simulators,  it  is  reasonable  to  assume  that  the  simulator  sickness  symptoms  the  pilots 
experienced  did  not  (1)  cause  them  significant  discomfort,  (2)  distract  them  during  missions,  or 
(3)  contribute  to  an  increase  in  perceived  workload. 

Table  9.  Comparison  of  RACRS  simulator  SSQ  ratings  with  other  helicopter  simulators. 


Nausea 

Oculomotor 

Disorientation 

Total  Severity  Score 

Simulator 

Subscale 

Subscale 

Subscale 

(Mean) 

AH-64A11 

— 

— 

— 

25.81 

ARH-70 

18.02 

21.48 

9.28 

20.15 

SH-3H 

14.70 

20.00 

12.40 

18.80 

RAH-66  EDS 

11.84 

14.98 

4.54 

13.25 

CH-53F 

7.50 

10.50 

7.40 

10.00 

RAH-66  CPC 

3.29 

12.94 

7.89 

9.80 

UH-60M  (LEUE) 

6.36 

11.81 

3.09 

9.15 

RACRS  (UAS) 

9.01 

7.58 

4.64 

8.51 

UH-60M  (EUD) 

13.88 

6.89 

0 

8.50 

CH-53D 

7.20 

7.20 

4.00 

7.50 

CH-46E 

5.40 

7.80 

4.50 

7.00 

RACRS  (VUIT-2) 

3.18 

5.05 

4.64 

4.98 

aSSQ  subscale  data  not  available. 
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5.  Conclusions 


Pilots  reported  that  they  typically  experienced  tolerable  workload  when  performing  missions 
while  controlling  the  UAS.  They  reported  that  the  workload  they  experienced  was  comparable 
to  workload  they  experienced  during  “non-VUIT-2”  missions.  They  stated  that  having  to 
interact  with  an  additional  sensor  (UAS  sensor)  increased  their  overall  task  workload,  but  the  SA 
provided  by  the  UAS  sensor  decreased  the  workload  required  to  detect  and  engage  targets,  and 
decreased  overall  target  engagement  timelines.  The  SMEs  reported  that  the  pilots  typically 
experienced  tolerable  workload  when  controlling  the  UAS  during  missions,  but  had  reduced 
spare  workload  capacity.  The  workload  ratings  provided  by  the  pilots  and  SMEs  were  lower 
than  the  Objective  and  Threshold  workload  ratings  requirements  listed  in  the  Apache  Block  III 
(AB3)  Capability  Development  Document  (CDD)  (table  10). 

5.1  Crew  Workload 

Table  10.  Pilot  workload  requirements  and  ratings. 


Pilot  Bedford  Workload 

SME  Bedford  Workload 

CDD  Bedford  Workload 

Ratings  for  VUIT-2 

Ratings  for  VUIT-2 

Rating  Requirements 

Missions 

Missions 

Objective  req.  -  ‘5.0’ 

PI -‘2.6’ 

PI -‘2.5’ 

Threshold  req.  -  ‘6.0’ 

CPG-  ‘3.3’ 

CPG -‘3.9’ 

5.2  Crew  Situation  Awareness 

Pilots  typically  experienced  moderate  levels  of  SA  during  missions.  They  reported  that  they  had 
high  levels  of  SA  of  most  of  the  battlefield  elements  (e.g.,  threat  location)  during  the  missions. 
However,  there  were  instances  when  they  flew  near  the  targets  and  fired  missiles  outside  of  the 
aircraft/UAS  constraints.  This  was  likely  caused  by  the  lack  of  extensive  training  and  experience 
with  the  UAS  and  the  need  for  improved  cueing  symbology  to  help  pilots  understand  where  their 
aircraft  and  the  UAS  are  located  in  reference  to  the  targets.  The  pilots  stated  that  they  had  higher 
SA  during  VUIT-2  missions  (vs.  non-VUIT-2  missions)  mostly  because  of  the  “God’s  Eye”  view 
that  the  UAS  sensor  video  provided  during  missions.  The  UAS  sensor  video  also  gave  the  pilots 
good  SA  earlier  in  the  mission  (vs.  non-VUIT-2  missions)  because  they  often  received  the  video 
prior  to  (or  just  after)  take-off  of  their  aircraft.  The  SMEs  reported  that  the  aircrews  typically 
had  adequate  levels  of  SA. 
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5.3  Crew  Coordination 


The  SMEs  identified  no  significant  problems  with  aircrew  coordination  during  the  VUIT-2  and 
non-VUIT-2  missions.  The  SMEs  rated  aircrew  coordination  during  VUIT-2  missions  as 
“Excellent”  (15%),  “Good”  (31%),  or  “Average”  (54%).  The  SMEs  rated  aircrew  coordination 
during  non-VUIT-2  missions  as  “Excellent”  (17%),  “Good”  (33%),  or  “Average”  (50%). 

5.4  UAS-Crewstation  Interface 

Most  pilots  reported  that  the  UCI  was  usable,  but  needed  improvements  to  enhance  overall 
effectiveness.  They  reported  that  they  were  able  to  appropriately  operate  the  VUIT  Interface 
Panel  (VIP)  switches  and  keypad  to  accomplish  UAS  mission  tasks,  although  the  current 
configuration  was  not  considered  the  most  efficient  design.  Pilots  commented  that  the  number  of 
switch  actuations  per  mission  was  not  excessive.  They  also  felt  that  the  overall  design  of  the 
UCI  somewhat  hindered  them  from  interacting  with  the  UAS  in  a  timely  manner,  and  reported 
that  the  UCI  contributed  to  high  workload  during  specific  data  entry  procedures  and  symbology 
interpretation.  In  some  instances,  the  buttons  were  considered  too  small  to  manipulate  quickly, 
and  pilots  reported  that  remembering  their  appropriate  functions  was  difficult.  The  functionality 
of  the  keypad  also  caused  an  increase  in  time  and  workload  during  some  missions.  All  of  the 
pilots  reported  that  the  symbology  presented  was  difficult  to  understand. 

The  pilots  recommended  that  several  enhancements  should  be  made  to  improve  the  VUIT-2 
crewstation  interface  and  overall  operation  (appendix  E).  The  pilots  commented  that  with 
several  enhancements  and  more  experience  using  the  system,  most  of  the  increased  workload  and 
decreased  efficiency  could  be  improved. 

5.5  Simulator  Sickness 

Pilots  reported  that  they  typically  experienced  very  mild  to  mild  simulator  sickness  symptoms 
during  the  evaluation.  The  simulator  sickness  symptoms  the  pilots  experienced  did  not  (1)  cause 
them  significant  discomfort,  (2)  distract  them  during  missions,  or  (3)  contribute  to  an  increase  in 
perceived  workload. 


6.  Recommendations 


The  following  recommendations  are  made  to  enhance  the  overall  effectiveness  and  suitability  of 
the  UAS  VUIT-2  integration  into  the  AH-64D: 

•  Address  and  incorporate  the  recommended  improvements  (as  appropriate)  provided  by  the 
pilots. 

•  Use  the  Crew  Station  Working  Group  to  address  and  incorporate  the  recommended 
improvements. 
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•  Upgrade  the  RACRS  simulator  to  make  it  representative  of  the  AH-64D  Block  3  design  to 
enhance  future  simulations. 

•  Continue  using  the  RACRS  to  help  train  pilots,  and  refine  the  UCI  and  TTPs  for  the  Block 
2  UAS  integration  (VUIT). 

•  Maximize  the  amount  of  pilot  training  for  future  UAS  evaluations. 

•  Use  the  same  data  collection  methodology  (e.g.,  Bedford,  SART)  during  future  simulations 
and  tests  for  Apache  VUIT-2  teaming.  Standardizing  the  data  collection  methodology  will 
help  identify  changes  that  work  (e.g.,  changes  that  reduce  workload),  identify  areas  that 
still  need  more  work,  and  drive  continuous  incremental  improvements. 

•  Incorporate  VUIT-2  capability  into  the  Longbow  Crew  Trainers  to  aid  with  pilot  training. 
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Appendix  A.  Bedford  Workload  Rating  Scale  Scores  and  Pilot  Comments 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Bedford  Workload  Rating  Scale 


Workload  Description 


Rating” 


Was  workload 
satisfactory 
without  reduction  in 
spare  (workload)  capacity? 


Was  workload  tolerable 
for  the  task? 


NO 


Was  it  possible  to 
complete  the  task? 


Pilot  Decisions 


Workload  insignificant 

1 

Workload  low 

2 

Enough  spare  capacity  for  all 
desirable  additional  tasks 

3 

Task  abandoned.  Pilot  unable  to 
apply  sufficient  effort 


Insufficient  spare  capacity  for  easy 
attention  to  additional  tasks 

4 

Reduced  spare  capacity.  Additional 
tasks  cannot  be  given  the  desired 
amount  of  attention 

5 

Little  spare  capacity:  level  of  effort 
allows  little  attention  to  additional 
tasks 

6 

Very  little  spare  capacity,  but 
maintenance  of  effort  in  the  primary 
tasks  not  in  question 

7 

Very  high  workload  with  almost  no 
spare  capacity.  Difficulty  in 
maintaining  level  of  effort 

8 

Extremely  high  workload.  No  spare 
capacity.  Serious  doubts  as  to  ability 
to  maintain  level  of  effort 

9 

10 
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Flight  and  Mission  Tasks 


Seat 


Before  Takeoff  Checks 


Observing  NAI’s 


Target  Detection 


Target  Acquisition 


Target  Engagement 


Movement  To  Contact 


Actions  On  Contact 


Battle  Damage  Assessment 
And  Reporting 


Mission  Change 


Battle  Handover 


Tactical  Navigation 
(Contour/NOE) 

Communications  (TOC,  Wingman) 


NOE/Contour/Low  Level  Flight 


Maintain  Airspace  Surveillance 


VMC  Flight  Maneuvers 


Electronically  Aided  Navigation 


Terrain  Flight  Navigation 


Masking  and  Unmaskin 


Evasive  Maneuvers 


MTADS/PNVS  Operations 


Route  Recon 


Area  Recon 


Level  2  UAS  Control 


Data  Entry  Procedures 


VUIT  2  Mission 
Workload  Rating 


Front 


1.3 


3.0 


2.9 


2.5 


2.1 


2.3 


2.7 


Back 


1.2 


1.5 


Non  VUIT-2  Mission 
Workload  Rating 

Front 

Back 

1.0 

1.0 

3.0 

1.0 

3.3 

2.0 

3.7 

3.0 

2.3 

2.0 

3.0 

1.5 

2.5 

2.5 

2.0 

1.5 

2.0 

3.0 

3.0 

1.0 

1.5 

1.5 

3.0 

2.0 

1.0 

2.0 

3.0 

1.0 

1.5 

1.7 

2.0 

1.0 

1.0 

2.0 

2.0 

2.0 

2.0 

1.5 

3.0 

1.5 

3.0 

1.0 
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Engage  with  Hellfire 

1.9 

2.1 

1.7 

1.5 

Engage  with  2.75  Rockets 

1.5 

1.5 

1.0 

Engage  with  30mm  AWS 

2.3 

1.3 

1.7 

1.0 

Multi-ship  Operations 

2.4 

1.5 

2.0 

2.3 

Transmit  Tactical  Reports 

2.0 

1.5 

2.0 

2.0 

Identify  Major  US/Allied  and  Threat 
Equipment 

2.1 

1.6 

3.0 

3.0 

Information  Management  In  The 
Front  Seat  (CPG) 

3.3 

I 

I 

I 

I 

I 

2.3 

Information  Management  In  The 
Back  Seat  (Pilot) 

I 

I 

I 

I 

I 

2.6 

2.3 

Average  Ratings  for  ATM  Tasks 

2.3 

1.8 

2.2 

1.8 

Overall  Workload  Rating  For  The 
Mission  (Average) 

3.3 

2.6 

3.0 

3.7 

Pilot  Workload  Comments 

If  you  gave  a  workload  rating  of  ‘5’  or  higher  for  any  task  in  the  UAS  mission  column,  explain  why 
the  workload  was  high  for  the  task. 

Explanations  as  to  why  the  pilots  assigned  a  rating  of  ‘5  ’  or  higher  for  a  task 


(VUIT-2  Missions) 

Question  1: 

Target  Detection  and  Acquisition: 

•  Target  detection  and  acquisition.  With  VUIT  you  have  the  ability  to  see  the  target  and 
atmosphere  from  a  distance.  W/out  VUIT  we  depend  on  “most  of  the  time”  a  general  grid  from 
ground  units.  VUIT  a  definite  plus. 

•  Target  detection.  You  know  there  is  a  target  out  there,  but  actually  detecting  it  is  frustrating.  - 
Non  UAV  is  frustrating  but  you  don’t  know  there  is  a  target  there. 

•  Target  Detection.  Enroute  VUIT  is  a  great  asset  in  detecting  targets.  Once  you’re  in  the  fight  it’s 
too  busy  for  the  CPG  to  be  messing  with  the  VUIT.  It  turns  the  pilot  into  truly  alone  and  in  the 
fight. 
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•  Target  Detection.  We  know  there  is  a  target  there,  but  where  in  relation  to  our  A/C  and  altitude. 

Maintain  Aircraft/ Airspace  Surveillance: 

•  Maintain  A/C  surveillance.  A  lot  of  heads  down  in  the  front  seat  in  light  out  operations  it  is  a  big 
concern  for  obstacle  avoidance. 

•  Maintain  Airspace.  Heads  down  entire  flight,  could  not  spare  time  to  help  clear  A/C. 

•  Airspace  Surveillance.  Same  as  before,  only  one  set  of  eyes  outside  and  a  more  complex  mission. 

Data  Entry: 

•  Data  Entry.  If  you  place  another  piece  of  equipment  in  the  aircraft  it  requires  you  to  enter  more 
information,  but  the  VUIT  increases  your  SA  lOx  fold  which  decreases  your  workload. 

•  Data  entry.  Takes  so  much  time  to  enter  numbers,  then  if  you  mess  up,  no  backing  up,  I  had  to 
restart  the  page. 

Information  Management: 

•  Info.  Management.  A  lot  of  info  quick,  CPG  has  to  prioritize  what  must  happen  first  and  this 
does  not  leave  much  spare  attention  to  other  tasks. 

•  Information  management.  Was  able  to  keep  awareness,  but  due  to  the  VUIT  I  was  having  some 
problems  maintaining  the  UAV  location.  Had  to  depend  on  the  CPG  for  that. 

•  Information  Management  (CPG).  I  still  spend  too  much  time  inside  and  not  outside  gathering 
situation  awareness. 

•  Pilot  Info  Management.  More  complex  mission  and  the  use  of  VUIT-2  limits  how  quickly  we  can 
change  from  VUIT  to  VCR. 

Communication: 

•  Commo.  A  lot  more  target  activity  and  commo  requirement  for  this  type  of  mission. 

(Non-VUIT-2  Missions) 

Question  1: 


Target  Acquisition: 

•  Target  acquisition.  Was  a  little  more  difficult  compared  to  a  VUIT  engagement.  Actually  PID’ing 
the  correct  target. 

•  TGT  ACQ  -  due  to  the  fact  the  simulator  is  not  like  the  A/C. 


33 


Communication: 


•  Commo.  More  complex  mission  requires  more  communications  between  all  members  of  the 
flight. 

List  any  flight  and/or  mission  tasks  that  you  had  to  ask  your  crewmember  to  accomplish  because 
your  workload  was  too  high. 

(VUIT-2  Missions) 

Question  2: 

•  I  didn’t  need  anything  to  be  done  for  me.  However,  sensing  how  busy  the  CPG  was  with  the 
VUIT,  the  radios,  and  targeting.  I  took  a  lot  of  the  radio  traffic  on  as  well  as  some  engagements. 

•  Engagement  of  one  target  was  taken  over  by  PLT  gun  because  of  closeness  to  target. 

•  2  Hellfire  missiles  lost  due  to  lost  of  SA  and  crew  coordination  due  to  workload.  Increase  in 
workload  due  to  both  mission  type  and  some  VUIT  requirements. 
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Appendix  B.  Subject  Matter  Expert  Workload  Comments 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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If  you  assigned  a  workload  rating  of  ‘5’  or  higher  for  either  crewmember,  explain  why: 

(VUIT-2  Missions) 

TSC  WL1: 

•  Much  better  “SA”  than  yesterday  for  this  crew.  CPG  had  a  much  better  grasp  of  the  system. 

•  Good  mission. 

•  Had  to  be  prompted  to  come  out  of  freeze  on  the  VUIT-2.  Did  not  use  VUIT-2  map  for 
orientation.  Front  seat  couldn’t  move  sensor,  but  was  still  autotracking. 

•  Both  set  up  VUIT  video,  B/S  up  flight  page.  B/S  told  warlock  deviation  to  engage  ZSU. 

•  Crew  still  had  difficulty  operating  VUIT  system.  Crew  recorded  engagement  well.  No  reports  to 
warlock  that  they  were  going  to  engage. 

•  Because  of  CPG  unfamiliarity  with  Longbow  systems,  this  caused  greater  workload  when  VUIT- 
2  capability  was  added.  CPG  saturated  and  having  difficulty  including  additional  tasks. 

(Non-VUIT-2  Missions) 

TSC  WL1: 

•  Pilot  inside  helping  CPG  with  engagement,  almost  allowed  aircraft  to  descend  into  ground. 

•  Flew  constantly  enroute  at  106%.  Front  seater  had  difficulty  manipulating  the  MTADS  auto 
tracker. 

•  Front  seat  was  unaware  that  his  missile  and  laser  code  did  not  match.  ZSU  engagement.  30mm 
was  at  the  AZ  limit.  A/C  heading  302  TADS  heading  180. 

•  The  crew  at  times  had  difficulty  orienting  their  sensor  and  the  A/C  in  the  same  direction  to 
effectively  track  and  engage  targets. 

SME  Aircrew  Coordination  Comments: 

(VUIT-2  Missions) 

TSC  WL2: 

•  Good  coordination  for  weapons  engagements. 

Describe  any  problems  that  aircrews  had  with  situation  awareness: 

(VUIT-2  Missions) 

TSC  SA: 

•  Engagement  1  -  Good  geometry/good  remote  shot. 
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•  Engagement  2  -  CPG  intent  on  adding  information  and  allowing  TADS  to  hit  symbol  units. 
While  changing  between  UAS  video,  crew  was  sent  incorrect  data  causing  confusion.  Updated 
grid  with  wrong  identifier  (VD  instead  of  WD).  CPG  kept  up  screen  capture  mode  after  inserting 
grid,  not  required  and  ties  up  MPD. 

•  Crew  was  aware  of  their  position  and  the  UAS  position. 

•  Engagement  1  -  Good  engagement. 

•  Engagement  2  -  Remote,  good  geometry  for  shot,  target  destroyed. 

•  Engagement  1  -  Good  shot.  Good  use  of  UAS. 

•  Engagement  2  -  Good  SA  comparing  UAS  video  to  TADS  video.  Crew  should  be  cognizant  of 
getting  too  close  to  enemy  (do  not  overfly  target). 

•  Engagement  1  -  CPG  inserted  wrong  Freq.  not  received  UAS  video  slowing  the  process.  This  is 
not  he  first  time  this  CPG  has  inputted  wrong  Freq,  but  getting  better.  Both  crewmembers  have 
VUIT-2  video  up  all  the  time.  Recommend  using  VUIT-2  video/map  for  orientation,  especially 
during  autonomous  engagement. 

•  Engagement  2  -  Crew  read  back  Freq’s.  (good  technique)  good  engagements. 

•  Good  mission. 

•  Operator  knowledge  of  MTADS  caused  confusion  in  cockpit.  CPG  should  try  to  coordinated 
UAS  video  to  TADS  video  instead  of  inserting  grids.  Inserting  grids  takes  more  times  for 
engagement.  CPG  has  difficulty  coordinating  video  from  UAS  and  TADS  video.  Fired  missile 
out  of  constraints,  shot  wrong  target.  UAS  video  frozen  for  5  minutes  without  CPG  correction. 

(Non-VUIT-2  Missions) 

TSC  SA: 

•  CMWS  ineffective  at  altitudes  aircraft  at.  Aircraft  susceptible  to  SA  missile  threats.  Pilot  so  low 
and  inside  that  he  allowed  aircraft  to  come  within  17  feet  of  ground  contact. 

•  Friendly  casualty  in  objective.  Friendly  given  as  in  mangrove,  north  of  3a.  Crew  shot  friendly. 

•  Poor  MTADS  tracking  almost  caused  missile  miss  on  ZSU. 

Did  the  aircrew  complete  their  mission  objectives? 

(Non-VUIT-2  Missions) 

TSC  MSI: 

•  Friendly  was  shot. 

If  no,  why  weren ’t  the  mission  objectives  completed? 
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(Non-VUIT-2  Missions) 
TSC  MS2: 

•  Fratricide. 
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Appendix  C.  Situation  Awareness  Ratings  and  Comments 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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SA  RATINGS  (VUIT-2  Missions) 
Front  Seat 


‘Demand’  During  Missions 

Low  1 - 3 - 5 - 7 - 9 - 


‘Supply’  During  Missions 

Low  1 - 4 - 7 - 10 - 13 


‘Understanding’  During  Missions 

Low  1 - 3 - 5 - 7 - 9 - 


Overall  SART  Score:  19.0 


Back  Seat 


‘Demand’  During  Missions 

12.5 

Low  1 - 3 - 5 - 7 - 9 - 11 - 13- — i\5- 


‘Supply’  During  Missions 

Low  1 - 4 - 7 - 10 - 13 - 16 - 19— ■ 


‘Understanding’  During  Missions 

Low  1 - 3 - 5 - 7 - 9 - 11 - 13 - 15- 


Overall  SART  Score:  21.38 
SA  RATINGS  (Non  -  VUIT-2  Missions) 
Front  Seat 
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Demand  During  Missions 

Low  1 - 3 - 5 - 7 - 


‘Supply’  During  Missions 

Low  1 - 4 - 7 - 10- 


‘ Understanding’  During  Missions 

Low  1 - 3 - 5 - 7- — 


Overall  SART  Score:  18.0 


Battlefield  Elements  Ratings  (VUIT-2  Missions) 


Battlefield 

Elements 

Very  High 
Level  of 
Situation 
Awareness 

Fairly  High 
Level  of 
Situation 
Awareness 

Borderline 

Fairly  Low 
Level  of 
Situation 
Awareness 

Very  Low 
Level  of 
Situation 
Awareness 

Front 

Back 

Front 

Back 

Front 

Back 

Front 

Back 

Front 

Back 

Location  of 
Enemy  Units 

25% 

38% 

50% 

25% 

13% 

25% 

13% 

13% 

0% 

0% 

Location  of 
My  Aircraft 
During 
Missions 

38% 

38% 

50% 

50% 

0% 

13% 

13% 

0% 

0% 

0% 

Location  of 
Other  Aircraft 

In  My  Flight 

0% 

0% 

25% 

25% 

50% 

63% 

13% 

13% 

13% 

0% 

Route 

Information 
(ACPs,  BPs, 
EAs,  RPs,  etc.) 

25% 

25% 

38% 

63% 

25% 

0% 

13% 

13% 

0% 

0% 

Status  of  My 
Aircraft 
Systems  (e.g., 
fuel 

consumption) 

13% 

38% 

38% 

38% 

25% 

13% 

13% 

0% 

13% 

13% 
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Battlefield  Elements  Ratings  (Non  -  VUIT-2  Missions) 


Battlefield 

Elements 

Very  High  Level 
of  Situation 
Awareness 

Fairly  High 
Level  of 
Situation 
Awareness 

Borderline 

Fairly  Low 
Level  of 
Situation 
Awareness 

Very  Low 
Level  of 
Situation 
Awareness 

Front 

Back 

Front 

Back 

Front 

Back 

Front 

Back 

Front 

Back 

Location  of 
Enemy  Units 

0% 

33% 

33% 

67% 

67% 

0% 

0% 

0% 

0% 

Location  of 
My  Aircraft 
During 
Missions 

67% 

0% 

0% 

100% 

0% 

0% 

33% 

0% 

0% 

0% 

Location  of 
Other  Aircraft 
In  My  Flight 

0% 

0% 

67% 

33% 

33% 

67% 

0% 

0% 

0% 

0% 

Route 
Information 
(ACPs,  BPs, 
EAs,  RPs, 
etc.) 

0% 

33% 

33% 

33% 

67% 

33% 

0% 

0% 

0% 

0% 

Status  of  My 
Aircraft 
Systems  (e.g., 
fuel 

consumption) 

33% 

0% 

33% 

67% 

33% 

33% 

0% 

0% 

0% 

0% 

Pilot  Situational  Awareness  Comments 
APACHE  UAS  SA  Comments 
SA2  (VUIT-2  Missions): 

•  When  the  grid  passed  off  by  the  battle  captain  was  wrong,  but  the  UAV  feed  cleared  up  the 
situation. 

•  Received  no  information  on  civilian  areas  but  also  didn’t  ask.  I  think  as  a  P/C  in  the  real  world,  I 
would  take  longer  before  allowing  the  CPG  to  shoot.  I’d  have  him  looking  at  collateral  damage 
possibilities  more.  We  had  problems  with  getting  correct  grids  for  targets  which  is  actually  more 
realistic.  The  terrain  was  challenging  (complicated  more  by  the  grid  issues).  Higher  altitudes 
helped  reduce  workload  immensely  and  kept  me  as  the  backseater  from  having  the  urge  to  get 
close  because  I  wouldn’t  be  able  to  see  the  target  due  to  structure  of  the  A/C. 

•  During  the  remote  engagements,  I  had  to  put  more  attention  to  finding  the  UAVs  location  to  me 
and  the  target. 
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Location  of  friendly  units.  Need  a  BFT  icon  for  UAV  to  provide  location  info  to  B/S.  F/S  has 
same  degree  of  UAS  location  on  VUIT  map  which  was  not  available  to  pilot  in  B/S. 


•  Once  Blackjack  dismounted  they  entered  the  city.  Commander  didn’t  know  where  his  soldiers 
were,  so  we  quit  shooting  until  he  said  they  were  clear.  We  got  too  close  and  would  have  taken 
damage,  however  based  on  the  speed  the  lift  and  ground  forces  moved  it’s  impossible  to  stay 
back.  A  lot  of  questions  that  an  AMR  with  lift  and  inf  would  have  corrected. 

•  When  the  troop  got  off  of  their  A/C,  I  could  not  see  them.  I  don’t  know  if  it  was  a  LCT  issue  or  if 
they  went  behind  a  building.  I  could  not  fire  due  to  unknown  location  at  first. 

•  I  didn’t  keep  up  with  the  other  A/C  once  we  engaged.  I  still  had  no  time  to  monitor  A/C  systems 
due  to  other  activities. 

•  Location  of  my  A/C  -  because  of  requirement  to  focus  inside  cockpit,  location  SA  was  lost  a  few 
times.  Majority  of  the  dwell  time  inside  cockpit  caused  by  similar  limitation  (no  IHADSS 
boresight)  some  caused  by  CPG  lack  of  familiarity  with  MTADS  (only  3.0  hours). 

•  Difficulty  remembering  to  unfreeze  UAV  feed. 

SA2  (Non  -  VUIT-2  Missions): 

•  I  was  inside  so  much  I  could  not  keep  a  mental  picture. 

•  We  were  uncertain  of  the  location  of  the  non  combatants  and  the  informant. 

•  Miscommunication  in  locating  personnel. 
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Appendix  D.  VUIT-2-Crewstation  Interface  Ratings  and  Comments 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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P 1 .  Did  you  experience  any  problems  using  the  VIP  keys  during  mission  to  accomplish  UAV  tasks? 
Place  an  X  in  the  ‘Yes’  or  ‘No’  column  for  each  key  listed  below.  If  you  checked  ‘Yes’,  explain  the 
problems  you  experienced: 


VIP  Keys 

Yes 

No 

1  -  UAV  MAP 

67% 

33% 

2  -  MTCDL  PG 

17% 

83% 

3  -  BIT  PG 

0% 

100% 

4  -  FRQ  SCRL 

67% 

33% 

5  -  STR  PIC 

0% 

100% 

6  -  RCL  PIC 

17% 

83% 

7  -  ZM  IN 

17% 

83% 

8  -  KEY  INFO 

0% 

100% 

9  -  ZM  OUT 

0% 

100% 

0  -  XMT  0/0 

17% 

83% 

CNCL  * 

33% 

67% 

NUM  LOC  # 

33% 

67% 

UAV  MAP 

•  Needed  smaller  icons.  My  icon  needs  same  cues.  Can’t  see  UAVS  holding  cues. 

•  Map  and  icons  are  hard  to  read  and  see. 

•  Concerned  that  the  system  will  no  be  able  to  hold  enough  map  data. 

•  Detail  is  limited. 

MTCDL  PG 

•  Just  needs  faster  means  of  fixing  Freq. 

FRQ  SCRL 

•  Just  need  the  ability  back  space  or  entry. 

•  Can’t  back  out. 

•  Needs  to  have  a  data  entry  field  of  its  own,  instead  of  defaulting  to  data  entry  field  on  key  2  page. 

•  Labeled  wrong,  should  be  UAV  pg. 

XMT  O/O 
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•  This  should  be  an  enter  also. 


ZM  IN 

•  It  zoomed  in  on  the  same  map,  not  different  scales. 

RCL  PIC 

•  Need  ability  to  delete  saved  images  that  are  no  longer  required. 

CNCL 

•  Need  single  space  clear  and  backspace  option  available. 

•  Need  to  have  a  single  back  clear,  not  clear  all  as  it  is  now. 

NUM  LOC 

•  Name  and  visual  does  not  really  correlate  to  the  buttons  function  which  is  cursor  select  and  data 
enter. 

•  Confusing  why  it  is  labeled  NUM  LOC. 

P2.  Overall,  how  quickly  were  you  able  to  use  the  VIP  keypad  and  switches  to  accomplish  UAV  tasks 
(e.g.,  data  entry,  toggling  keys)? 


2.7 

1 

2 

3 

4 

5 

' r 

Very 

Quickly 

Somewhat 

Quickly 

Borderline 

Slowly 

Somewhat 

Slowly 

Very 

If ‘Somewhat  Slowly  or  ‘Very  Slowly’,  describe  the  problems  you  experienced: 

•  The  buttons  are  too  small  for  fast  operation.  There  needs  to  be  more  streamline  setup  like  a 
computer  keyboard.  “Just  use  the  KU”  keyboard  unit  in  the  A/C  now. 

•  Rated  “Somewhat  Slowly”  due  to  operator  proficiency.  I  expect  to  improve  with  continued  use. 
A  second  reason  is  the  button  functionality  issues  addressed  on  page  1  i.e.  cncl  clear  function, 
key  4  data  entry  on  key  2  page,  etc. 

P3.  Was  there  any  UAV  symbology  or  wording  depicted  on  the  MPD  that  was  difficult  to  quickly  and 
easily  understand? 

•  Yes  100%  No  0% 
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If  yes,  explain  which  symbology  or  wording  was  difficult  to  understand  and  why: 

•  Hard  to  see  icon  cues  on  map. 

•  The  symbology  was  very  hard  to  see  on  our  MPD.  The  icons  represented  around  800  meters  on 
the  map  scale,  which  we  used.  It  made  it  hard  to  tell  where  everything  was  really  at.  You  could 
not  see  the  symbology  which  tells  which  way  it  was  flying  or  looking. 

•  Lack  of  flight  info  (i.e.  north  arrow,  UAS  altitude,  uas  speed,  uas  direction  of  travel)  for  UAS. 
Lack  of  color  symbols  (i.e.  sensor  pointing  direction)  displayed  on  map. 

•  LOS  arrow  symbology  should  be  black,  not  white  and  have  a  different  symbol  on  the  end.  — X 

•  The  UAV  line  of  sight  arrow  is  difficult  to  break  out  from  the  background  map. 

•  Symbology  on  the  map  page  is  too  large  for  that  scale  of  map.  Need  to  have  a  North  seeking 
arrow  on  VUIT  video  page. 

P4.  On  average,  how  often  did  the  design  of  the  UAV  interface  (e.g.,  VIP,  infor  on  MPD)  significantly 
hinder  you  from  quickly  and  easily  performing  UAV  tasks? 

(Circle  one) 


1 

2 

2.5 

3 

A 

' ' 

i 

Never 

Seldom 

Often 

Frequently 

•  If  ‘Often  or  Frequently’,  describe  how  the  design  of  the  UAV  interface  significantly  hindered  you 
from  quickly  and  easily  performing  UAV  tasks: 

•  Its  just  hard  because  its  not  integrated  and  your  MPD  usage  is  too  much. 

•  Same  as  similar  problems. 

•  Not  having  a  back  arrow  or  a  dedicated  “enter”  button  slows  down  our  ability  to  quickly  input  the 
desired  info. 

P5.  On  average,  how  often  did  the  design  of  the  UAV  interface  contribute  to  high  workload  when 
controlling  the  UAV?  (Circle  one) 


2.7 

1 

2 

3 

4 

' r 

Never 

Seldom 

Often 

Frequently 
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If  ‘Often  or  Frequently’,  describe  how  the  design  of  the  UAS  interface  contributed  to  high  workload 
during  missions: 

•  It  was  the  whole  problem. 

•  Same  as  similar  problems 

•  Not  having  a  back  arrow  or  a  dedicated  “enter”  button  slows  down  our  ability  to  quickly  input  the 
desired  info. 

P6.  List  any  other  UAV  interface  usability  features  that  hindered  your  performance  during  the 
missions: 

•  Not  being  able  to  record  your  video  at  same  time. 

•  Most  important  is  inability  to  record  VUIT  video  and  coordinating  communications  with  the  on 
board  A/C  recording  system. 
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Intentionally  Left  Blank. 
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Appendix  E.  Top  Improvements  for  VUIT-2  Integration 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Top  Improvements  Recommended  By  Pilots 
Video/VCR 

•  Need  to  be  able  to  record  on  VCR  as  we  receive  VUIT  video. 

•  Redesign  to  record  UAV  feed. 

•  Have  two  recorders  so  we  don’t  have  to  switch  ours  off. 

•  Need  to  be  able  to  record  our  TADS  video  without  switching.  People  will  forget  to  switch 

•  On  board  recording  capability  of  VUIT  video  and  comms. 

Symbology 

•  Need  to  change  the  colors  of  the  icon  so  we  can  scan  them. 

•  LOS  arrow  needs  to  better  break  itself  out  from  the  background. 

•  Change  MPD  UAV  symbol  to  black. 

TSD  Integration 

•  Needs  to  work  with  current  TSD  page. 

•  Have  the  UAV  icon  integrated  on  our  TSD. 

Map 

•  Need  to  see  our  site  and  heading  on  the  map  icon.  Very  important  for  remote.  This  will  reduce 
the  time. 

•  Map  data  fidelity. 

•  Need  to  have  the  ability  of  different  scales  of  maps  in  the  VUIT  feed.  Making  the  symbols  scale 
to  each  map.  Current  symbols  are  too  large  for  the  1 :250k  map. 

Keyboard/Button  Functionality 

•  Make  sure  the  VIP  works  the  same  as  the  keyboard  unit.  Need  an  enter  button,  and 
backspace/clear  button. 

•  Button  functionality  issues. 

•  Have  a  backup  button  when  your  inputting  frequency. 

•  Need  to  have  an  “enter”  button  and  a  single  space,  backspace  ability. 

•  Use  a  rocker  switch  for  scroll  and  zoom,  this  will  free  up  a  couple  of  buttons  for  other  functions. 

Miscellaneous 
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•  Being  able  to  recall  the  MGRS  grid  after  you  have  stored  it. 

•  Get  a  standard  format  for  the  remote  handover. 

•  Be  able  to  select  a  preset  frequency. 
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Intentionally  Left  Blank. 
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Appendix  F.  Switch  Actuations  (UAS) 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Number  of  Switch  Actuations  Per  Mission 

Column 

Detailed  Description 

1 

2 

3 

4 

5 

6 

7 

8 

Avg 

Std  Dev 

Max 

TFOV 

Left  Hand  Grip  TAD  FOV  Select 

66 

54 

24 

20 

153 

130 

80 

81.625 

50.04551 

153 

Slave 

Right  Hand  Grip  Slave  Select 

43 

30 

16 

26 

68 

83 

53 

43 

23.4094 

83 

Tracker 

Left  Hand  Grip  Tracker  Switch 

8 

38 

4 

10 

18 

44 

46 

36 

37.85687 

120 

Laser 

Right  Hand  Grip  Laser  Trigger 

24 

21 

6 

3 

41 

51 

64 

33.75 

23.68695 

64 

CPGR 

CPG  Right  Bezel  Buttons 

43 

44 

3 

38 

20 

29 

35 

29.125 

13.93287 

44 

PLTL 

Pilot  Left  Bezel  Buttons 

52 

11 

8 

28 

24 

15 

4 

7 

18.625 

15.87395 

52 

PLTR 

Pilot  Right  Bezel  Buttons 

45 

6 

22 

10 

25 

0 

21 

7 

17 

14.38253 

45 

CPGL 

CPG  Left  Bezel  Buttons 

32 

16 

39 

5 

21 

0 

6 

8 

15.875 

13.8918 

39 

WpnTrig 

Weapon  Trigger 

4 

4 

4 

4 

20 

12 

46 

26 

15 

15.11858 

46 

WpnSel 

Left  Hand  Grip  Weapon  Select 

6 

6 

2 

20 

20 

14 

8 

12 

11 

6.676184 

20 

TfitSt 

Left  Hand  Grip  Target  Store 

4 

5 

0 

0 

12 

10 

0 

37 

8.5 

12.39816 

37 

LMC 

Left  Hand  Grip  LMC  Select 

3 

8 

0 

4 

1 

6 

0 

10 

4 

3.741657 

10 

Sensor 

Left  Hand  Grip  Sensor  Select 

2 

5 

0 

2 

0 

0 

0 

2 

1.375 

1.767767 

5 

DisB 

TEDAC  Display  Brightness  Rocker  Switch 

0 

2 

0 

2 

0 

0 

0 

1 

0.625 

0.916125 

2 

TVidSel 

TEDAC  TADS  Video  Select 

0 

0 

0 

0 

0 

0 

0 

2 

0.25 

0.707107 

2 

Level 

TEDAD  Level  Rocker  Switch 

0 

0 

0 

0 

0 

0 

0 

1 

0.125 

0.353553 

1 

DisC 

TEDAC  Display  Contrast  Rocker  Switch 

0 

0 

0 

0 

0 

1 

0 

0 

0.125 

0.353553 

1 

UFOV 

UAV  Field  of  View 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Gain 

TEDAC  Gain  Rocker  Switch 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

PVidSel 

TEDAC  PNVS  Video  Select 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

SymB 

TEDAC  Symbology  Brightness  Rocker  Switch 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Scan 

Left  Hand  Grip  Scan  Switch 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

UVidSel 

TEDAC  UAV  Video  Select 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Map 

Right  Hand  Grip  Map  Symbols  Select 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Sight 

Right  Hand  Grip  Sight  Select 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Enter 

Right  Hand  Grip  Enter  for  Cursor  Control 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Polarity 

Right  Hand  Grip  Polarity  Select 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Sum 

332 

250 

128 

172 

423 

395 

437 

391 

316 

118.7121 

437 

Time  of  Movement 

Column 

Detailed  Description 

1 

2 

3 

4 

5 

6 

7 

8 

Avg 

Std  Dev 

Max 

TFCAz 

RHG  Thumb  Force  Controller  Azimuth 

396.7 

374.2 

202.3 

100.2 

880.5 

494.9 

685.2 

915.7 

506.2125 

299.181 

915.7 

TFCEI 

RHG  Thumb  Force  Controller  Elevation 

369.6 

360.1 

217.5 

113 

877.1 

475.3 

646.5 

860.9 

490 

282.8116 

877.1 

Cursor  UD 

Left  Hand  Grip  Cursor  Up/Down 

0 

0 

0 

0 

0 

13.115 

37.08674 

Cursor  RL 

Left  Hand  Grip  Cursor  Right/Left 

0 

0 

0 

0 

0.00375 

0.005175 

VUIT  -  2 

1 

2 

3 

4 

5 

6 

7 

8 

Avg 

Std  Dev 

Max 

Button  Presses 

60 

66 

33 

40 

36 

73 

65 

44 

52.125 

15.56037 

73 

Combined  Total  of  Button  Presses 

392 

316 

161 

212 

459 

468 

502 

435 

368.125 

126.0334 

502 

Switch  Actuations  (Non  -  UAS) 


Number  of  Switch  Actuations  Per  Mission 

Column 

Detailed  Description 

1 

2 

3 

4 

Avg 

Std  Dev 

Max 

TFOV 

Left  Hand  Grip  TAD  FOV  Select 

224 

140 

126 

196 

171.5 

46.25653 

224 

Tracker 

Left  Hand  Grip  Tracker  Switch 

68 

86 

26 

123 

75.75 

40.30199 

123 

Laser 

Right  Hand  Grip  Laser  Trigger 

43 

48 

43 

70 

51 

12.8841 

70 

Slave 

Right  Hand  Grip  Slave  Select 

81 

47 

36 

26 

47.5 

23.92349 

81 

WpnTrig 

Weapon  Trigger 

24 

54 

12 

46 

34 

19.39072 

54 

Level 

TEDAD  Level  Rocker  Switch 

46 

30 

4 

1 

20.25 

21.54646 

46 

CPGR 

CPG  Right  Bezel  Buttons 

12 

28 

18 

15 

18.25 

6.946222 

28 

Gain 

TEDAC  Gain  Rocker  Switch 

30 

41 

0 

0 

17.75 

20.98214 

41 

WpnSel 

Left  Hand  Grip  Weapon  Select 

20 

12 

18 

20 

17.5 

3.785939 

20 

CPGL 

CPG  Left  Bezel  Buttons 

20 

2 

32 

9 

15.75 

13.1244 

32 

TgtSt 

Left  Hand  Grip  Target  Store 

5 

13 

29 

10 

14.25 

10.37224 

29 

PLTL 

Pilot  Left  Bezel  Buttons 

34 

1 1 

6 

0 

12.75 

14.86327 

34 

LMC 

Left  Hand  Grip  LMC  Select 

10 

0 

28 

3 

10.25 

12.55322 

28 

PLTR 

Pilot  Right  Bezel  Buttons 

7 

3 

12 

7 

7.25 

3.685557 

12 

TVidSel 

TEDAC  TADS  Video  Select 

3 

13 

0 

0 

4 

6.164414 

13 

Sensor 

Left  Hand  Grip  Sensor  Select 

4 

0 

6 

0 

2.5 

3 

6 

SymB 

TEDAC  Symbology  Brightness  Rocker  Switch 

0 

8 

0 

0 

2 

4 

8 

PVidSel 

TEDAC  PNVS  Video  Select 

0 

5 

0 

0 

1.25 

2.5 

5 

DisB 

TEDAC  Display  Brightness  Rocker  Switch 

0 

1 

1 

0 

0.5 

0.57735 

1 

DisC 

TEDAC  Display  Contrast  Rocker  Switch 

0 

0 

1 

1 

0.5 

0.57735  1 

Sight 

Right  Hand  Grip  Sight  Select 

2 

0 

0 

0 

0.5 

1 

2 

Polarity 

Right  Hand  Grip  Polarity  Select 

0 

1 

0 

0 

0.25 

0.5 

1 

UFOV 

UAV  Field  of  View 

0 

0 

0 

0 

0 

0 

0 

Scan 

Left  Hand  Grip  Scan  Switch 

0 

0 

0 

0 

0 

0 

0 

UVidSel 

TEDAC  UAV  Video  Select 

0 

0 

0 

0 

0 

0 

0 

Map 

Right  Hand  Grip  Map  Symbols  Select 

0 

0 

0 

0 

0 

0 

0 

Enter 

Right  Hand  Grip  Enter  for  Cursor  Control 

0 

0 

0 

0 

0 

0 

0 

Sum 

633 

543 

398 

527 

525.25 

96.81727 

633 

ON 

o 


Time  of  Movement 

Column 

Detailed  Description 

1 

2 

3 

4 

Avg 

Std  Dev 

Max 

TFCAz 

RHG  Thumb  Force  Controller  Azimuth 

983.4 

820.4 

1314.6 

1061.8 

1045.05 

205.9175 

1314.6 

TFCEI 

RHG  Thumb  Force  Controller  Elevation 

961.8 

796.6 

1214.5 

1034.8 

1001.925 

173.2409 

1214.5 

Cursor  UD 

Left  Hand  Grip  Cursor  Up/Down 

18.5 

0 

0 

0 

4.625 

9.25 

18.5 

Cursor  RL 

Left  Hand  Grip  Cursor  Right/Left 

0.02 

0 

0 

0.01 

0.0075 

0.009574 

0.02 

Time  and  Percentage  of  Time  That  Menu  Pages  Were  Displayed  on  MPDs 

UAS  Missions 


Title 

Fit  Page 
VCR  Page 
Video  Page 
Eng  Page 
Wpt  Page 
Test  Status  Page 


Point  Page 
ABR  Page 
Perf  Page 
Wpn  Util  Page 
Show  Page 
TSD  Page 
Fuel  Page 
Load  Page 
Freq  Page 


Pilot 

(Left  MPD) 


Time  (s) 
4604.1 

4413.8 
3117 

1988.8 

1523.9 
669.6 


555.2 

254.3 

235.5 

210.9 
39.7 

26.9 

24.5 
12.4 
11.2 


Title 

Wpt  Page 
Video  Page 
VCR  Page 
Test  Status  Page 
Route  Menu  Page 
Eng  Page 


Point  Page 
Show  Page 
Fit  Page 
AC  Util  Page 
ABR  Page 
Ctrlm  Page 
TSD  Page 


Perf  Page 


Tre  Page 


Pilot 

(Right  MPD) 


Time  (s) 

7497.2 

5661.3 
2699.1 

771.2 

539.2 
252.4 


54.8 

46.7 

40.8 

33.7 
31.6 

22.8 
12.8 


12.7 


12 
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Non-VUIT-2  Missions 


Title 

Test  Status  Page 
Eng  Page 
Fit  Page 


Point  Page 
Video  Page 
AC  Util  Page 
VCR  Page 
Show  Page 
Tre  Page 
Fuel  Page 


WCA  Page 
Sys  Page 


CPG 

(Left  MPD) 
Time  (s) 

5993.3 

835.4 

709 


•  “S'- 

ASE  Page 

105.9 

v/  /v 

1% 

Load  Page 

85 

1% 

TSD  Page 

17 

<1% 

Wpt  Page 

13 

<1% 

Title 

Wpt  Page 
Route  Menu  Page 
DTU  Page 
Test  Status  Page 
Ctrlm  Page 
Load  Page 


CPG 

(Right  MPD) 
Time  (s) 
7573.4 

335.8 

327.5 
160.3 

100.5 

68.8 
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Appendix  G.  Operations  Order 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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1.  Situation: 


a)  Enemy  Forces 

1)  Situation 

a.  Enemy  -  Enemy  forces  are  operating  all  across  the  battle  space.  Enemy  forces  are 
loosely  organized  into  small  teams  which  act  independently  in  support  of  a  larger 
objective  of  destabilizing  the  local  government  and  swaying  the  local  populace 
against  allied  forces. 

b.  Weather  -  clear,  winds  are  180/5  G10,  vis  >7  mile. 

2)  Capabilities  -  These  teams  are  normally  made  up  of  local  20  -  40  year  old  men  with  the 
likely-hood  of  a  non-local  formally  trained  team  leader.  Their  weapons  consist  of  small 
to  medium  caliber  weapons,  mortars,  rockets,  SA7s  or  16s,  mines  and  IEDs.  They 
normally  will  go  to  ground  in  the  event  of  attack  but  are  not  reluctant  to  stand  and  fight 
if  their  home  or  leader  is  threatened.  Sympathetic  boarder  nations  have  been  supplying 
the  enemy  forces  with  weapons  that  are  becoming  increasingly  lethal.  The  border 
nations  have  also  pre-positioned  small  armored  units  near  the  boarder  as  a  show  of 
force  and  to  deter  allied  nations  from  cross-border  pursuit.  The  units  have  been  known 
to  surge  forward  toward  the  border  to  provoke  a  response  from  allied  forces  but  have 
stopped  short  of  crossing  the  international  boundary. 

3)  Probable  course  of  action  -  Their  most  probable  course  of  action  is  to  ambush  allied 
forces  using  mines,  IEDs  and  SAMs  followed  by  small  arms  fire  with  a  quick  retreat 
before  allied  re-enforcements  arrive.  Their  most  dangerous  course  of  action  is  to 
ambush  allied  forces  using  mines,  IEDs  and  SAMs  followed  by  a  deliberate  defense  of 
the  target  area  to  delay  allied  forces.  This  may  be  supported  by  a  cross  border  incursion 
of  the  border  nation  to  aid  in  the  destruction  of  the  allied  forces  or  withdraw  of  the  local 
enemy  insurgent  back  to  the  sanctuary  of  the  border  nation. 

b)  Friendly  Forces. 

1)  Mission  of  Next  higher  unit.  -  1  -  14th  Aviation  Regiment  mission  is  to  support  3BCT 
with  attack  helicopter  support  to  prevent  or  deter  enemy  forces  from  disrupting  3  BCTs 
movement. 

2)  Mission  of  adjacent  units  -  1st  and  2d  BCT  are  conducting  similar  missions  in  the  city 
and  plains  sector  of  the  battle  space. 

3)  Mission  and  location  of  supporting  elements 

a.  HHC  1  -  14th  (Heavy)  is  located  at  the  1  -  14th  Assembly  Area  with  the  mission  to 
provide  class  3/5. 
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b.  2- 14th  (Warlock)  is  located  at  the  1-1 4th  Assembly  Area  with  the  mission  to 
provide  lift  capability  to  3BCT. 

c.  JSTARS  is  on  station  as  directed  to  provide  over-watch  and  Intel  on  NAIs  as 

directed. 

d.  A  Co  UAV  (Shadow)  is  located  at  1  -  14th  Assembly  Area  with  the  mission  to  provide 

over-watch  and  Intel  for  the  3BCT  and  supporting  elements  as  directed  by  3  BCT. 

e.  1-1 5th  In  (Blackjack)  is  located  at  the  1  -  14th  Assembly  Area  with  the  mission  to 
prevent  insurgent  activity  in  their  area  of  operation  and  to  destroy  insurgent  forces 
operating  in  their  battle  space. 

2.  Mission: 

B  Company  1-1 4th  Aviation  Regiment  will  conduct  route  and  area  recon  in  support  of  2- 14th  Air 
Assault  into  objective  Red  to  prevent  hostile  engagement  of  friendly  forces.  On  order,  conduct 
Close  Combat  Attack  to  destroy  enemy  forces  in  objective  red  to  prevent  the  delay  of  1-1 5th 
Infantry’s  advance  into  the  objective.  Be  prepared  to  conduct  a  hasty  attack  to  destroy  units 
moving  from  the  border  nation  in  order  to  prevent  their  influence  on  the  objective. 

3.  Execution: 

a)  Concept  of  Operation.  Will  be  an  Air  Assault  500m  North  of  objective  red  by  1-1 5th 

Infantry  with  recon  and  over- watch  provided  by  B  Company  1  -  14th  Once  on  the  ground  1- 
15th  will  advance  into  objective  red  to  destroy  enemy  forces. 

1)  Scheme  of  maneuver.  B  1-1 4th  will  depart  with  2-14  Avn  Assets  to  conduct  route  and 
area  recon.  Once  at  the  objective,  B  1-14  will  set-up  in  an  over-watching  position  of  the 
objective  to  provide  fires  for  1-1 5th  infantry.  B  1-1 4th  will  then  conduct  relief  on 
stations  to  provide  continuous  over  watch  of  the  objective  until  released  by  1-1 5th  or 
higher  headquarters. 

2)  Formation.  Teams  abreast 

3)  Route.  Ingress  -  Route  Falcon 
Egress  -  Route  Sparrow 

4)  Tactical  Missions  to  subordinate  Units. 

a)  A  Company  UAV  -  will  depart  prior  to  B  1-1 4th  departure  to  have  the  UAVs  on 
station  at  vicinity  of  Waypoint  2  and  LZ.  A  UAV  will  provide  route  and  area  recon 
while  another  recons  the  LZ  and  Objective  Red. 

b.  3- 17th  FDC  -  will  provide  priority  fires  to  1-1 5th  and  B  1  -  14th  Aviation  for  the 
entirety  of  the  mission. 
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5)  Coordinating  Instructions.  B  Company  l-14th  and  A  Company  UAS  will  conduct  a 
commo  check  prior  to  departure  and  a  UAS  video  link-up  when  within  range  of  the 
UAV  in  the  vicinity  of  Waypoint  2  (CC).  Additional  control  will  be  governed  by  the 
direction  of  the  1-1 5th  ground  commander. 

4.  Service  Support: 

a)  Ammunition  load  is  300  rounds  of  30mm,  12  rockets  and  8  Hellfires. 

b)  Fuel  is  2+30. 

5.  Command  and  Signal.  See  Communications  Card 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


AAR 

after-action  review 

AB3 

Apache  Block  III 

ACP 

Air  Control  Point 

AH 

Attack  Helicopter 

AOI 

area  of  interest 

ARH 

Armed  Recon  Helicopter 

ARL 

U.S.  Army  Research  Lab 

ASL 

Applied  Science  Laboratories 

ATM 

Aircrew  Training  Manual 

BWRS 

Bedford  Workload  Rating  Scale 

CDD 

Capability  Development  Document 

CONOPS 

Concept  of  Operations 

CPC 

Comanche  Portable  Cockpit 

CPG 

copilot-gunner 

DIS 

distributed  information  system 

EDS 

Engineering  Development  Simulation 

EUD 

Early  User  Demonstration 

FAC 

Flight  Activity  Category 

FCR 

fire  control  radar 

FOV 

field  of  view 

HRED 

Human  Research  and  Engineering  Directorate 

ICS 

interphone  communication  system 

FEUE 

Limited  Early  User  Evaluation 

MPD 

multi-purpose  display 
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MTADS 

modernized  target  acquisition  and  designation  sight 

MTCDL 

mini-tactical  common  data  link 

NOE 

nap  of  the  Earth 

NVG 

Night  Vision  Goggles 

OneSAF 

One  Semi-Automated  Forces 

OSRVT 

one  system-remote  video  terminal 

PI 

pilot  (pilot  in  back  seat  who  flew  the  aircraft) 

PMO 

Product  Manager’s  Office 

PNVS 

pilot  night  vision  sensor 

RACRS 

Risk  and  Cost  Reduction  Simulator 

RL 

Readiness  Level 

SA 

situation  awareness 

SART 

Situational  Awareness  Rating  Technique 

SME 

subject  matter  expert 

SSQ 

Simulator  Sickness  Questionnaire 

TCDL 

Tactical  Common  Data  Link 

TEDAC 

TADS  Electronic  Display  and  Control 

TRADOC 

Training  and  Doctrine  Command 

TS 

Total  Severity 

TSD 

Tactical  Situation  Display 

TSMRA 

TRADOC  System  Manager,  Reconnaissence  Attack 

Tip 

tactics,  techniques,  and  procedures 

UAS 

unmanned  aerial  system 

UCI 

UAS  Crewstation  Interface 

UHF 

ultra  high  frequency 

VFR 

Visual  Flight  Rules 

VIP 

VUIT  Interface  Panel 
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VMC 

VUIT-2 

WSRT 


Visual  Meteorological  Conditions 

Video  from  UAS  for  Interoperability  Teaming  Level  II 

Wilcoxon  Signed  Ranks  Test 
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NO.  OF 

COPIES  ORGANIZATION 


1  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
only)  DTIC  OCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BEL  VOIR  VA  22060-6218 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  HRR 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  Cl  OK  TL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  Cl  OK  PE 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 


ABERDEEN  PROVING  GROUND 
1  DIR  USARL 

AMSRD  ARL  Cl  OK  TP  (BLDG  4600) 
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NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  ML  J  MARTIN 
MYER  CENTER  RM2D311 
FT  MONMOUTH  NJ  07703-5601 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MZ  A  DAVISON 
320  MANSCEN  LOOP  STE  1 15 
FT  LEONARD  WOOD  MO  65473 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MD  T  DAVIS 
BLDG  5400  RM  C242 
REDSTONE  ARSENAL  AL  35898-7290 

1  COMMANDANT  USAADASCH 
ATSA  CD 

AMSRD  ARL  HR  ME  DR  HAWLEY 
5800  CARTER  RD 
FT  BLISS  TX  79916-3802 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MM  DR  V  J  RICE 
BLDG  4011  RM  217 
1750  GREELEY  RD 
FT  SAM  HOUSTON  TX  78234-5002 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MG  R  SPINE 
BLDG  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
ARMC  FIELD  ELEMENT 
AMSRD  ARL  HR  MH  C  BURNS 
THIRD  AVE  BLDG  1467B  RM  336 
FT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 
AWC  FIELD  ELEMENT 
AMSRD  ARL  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MY  M  BARNES 
2520  HEALY  AVE 
STE  1172  BLDG  51005 
FT  HUACHUCA  AZ  85613-7069 


1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MP  D  UNGVARSKY 
POPE  HALL  BLDG  4709 
BCBL  806  HARRISON  DR 
FT  LEAVENWORTH  KS  66027-2302 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MJF  J  HANSBERGER 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PKWY  STE  B 
SUFFOLK  VA  23435 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MQ  M  R  FLETCHER 
NATICK  SOLDIER  CTR 
AMSRD  NSC  WSE  BLDG  3  RM  343 
NATICK  MA  01760-5020 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MT  J  CHEN 
12423  RESEARCH  PKWY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MT  C  KORTENHAUS 
12350  RESEARCH  PKWY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MS  C  MANASCO 
SIGNAL  TOWERS 
BLDG  29808A  RM  303 
FT  GORDON  GA  30905-5233 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MU 
6501  E  11  MILE  RD  MS  284 
BLDG  200A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MF  C  HERNANDEZ 
2421  NW  AUSTIN  RD  STE  220 
FT  SILL  OK  73503-9043 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MV 
S  MIDDLEBROOKS 
91012  STATION  AVE  RM  348 
FT  HOOD  TX  76544-5073 
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1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FT  BRAGG  NC  28310-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 
AMSRD  ARL  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FT  BENNING  GA  31905-5400 

1  ARMY  G1 
(CD  DAPEMR  B  KNAPP 
only)  300  ARMY  PENTAGON  RM  2C489 
WASHINGTON  DC  20310-0300 


ABERDEEN  PROVING  GROUND 


6  DIR  USARL 

AMSRD  ARL  Cl  OK  TP 
S  FOPPIANO 
AMSRD  ARL  HR 
P  FEDELE 
T  LETOWSKI 
AMSRD  ARL  HR  MD 
D  HARRAH 
AMSRD  ARL  HR  MR 
F  PARAGALLO 
AMSRD  ARL  HR  SD 
B  AMREIN 

TOTAL:  33  (1  PDF,  31  HCs,  1  CD) 
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